Image processing method, image processing apparatus, robot-mounted transport apparatus, and system

Abstract

The present invention is provided with a machine tool (10), a robot (25) which has a camera (31), and a conveyance device (35) on which the robot (25) is mounted, wherein an identifying figure is arranged within a machining area of the machine tool (10).

Description

Technical Field

[0001] The present disclosure relates to a robot-mounted transfer device including a machine tool for processing a workpiece, a robot for performing operations on the machine tool, and a transfer device for mounting and moving the robot, a system including these components, an image processing method, and an image processing apparatus.

Background Art

[0002] Conventionally, as an example of the above-described system, a system disclosed in Japanese Patent Application Laid-Open No. 2017-132002 (Japanese Patent Application Laid-Open Gazette) is known. In this system, an automated guided vehicle (AGV) equipped with a robot moves to a work position set for a machine tool, and at the work position, operations such as attaching and detaching a workpiece to and from the machine tool are performed by the robot.

[0003] In such a system, since a single robot moving by an AGV can perform operations such as attaching and detaching a workpiece to and from a plurality of machine tools, compared to a case where a robot is disposed in a fixed state with respect to a machine tool, the degree of freedom in the layout of the machine tools increases. Therefore, the layout of the machine tools can be set to a layout that can further improve production efficiency. Also, compared to a conventional system in which a robot is disposed in a fixed state, since a single robot can perform operations on more machine tools, the equipment cost can be reduced.

[0004] On the other hand, since the AGV has a structure that self-propels using wheels, the positioning accuracy when it stops at the work position is not necessarily high. Therefore, in order for the robot to perform accurate operations on the machine tool, it is necessary to compare the posture of the robot when the AGV is positioned at the work position with the reference posture of the robot set during so-called teaching, which is a control reference, detect the error amount, and correct the working posture of the robot according to the error amount.

[0005] As a technique for correcting the posture of such robots, a position correction method, such as the one disclosed in Japanese Patent Application Publication No. 2016-221622, is known. Specifically, this position correction method involves arranging a visual target consisting of two calibration markers on the outer surface of a machine tool, capturing an image of the visual target with a camera installed on the movable part of the robot, measuring the relative positional relationship between the robot and the machine tool based on the obtained image and the position and orientation of the camera, and correcting the robot's working posture based on the measured positional relationship. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2017-132002 [Patent Document 2] Japanese Patent Publication No. 2016-221622 [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] However, with the conventional position correction methods described above, it was not possible to accurately correct the posture of the robot performing the attachment and detachment operation, for example, when inserting a robot's hand into a machine tool and using that hand to attach and detach a workpiece to or from the machine tool's chuck.

[0008] In other words, since the automated guided vehicle is configured to move by the movement of wheels with a relatively high degree of freedom, the mounting surface on which the robot is mounted tends to tilt relative to the floor surface, and this tilt tends to fluctuate in accordance with changes in the posture of the mounted robot, or in other words, in accordance with changes in the center of gravity of the robot.

[0009] Therefore, when the robot takes the position in which its hand enters the machine tool during the workpiece attachment and detachment described above, in other words, when the robot's arm is significantly overhanging from the automated guided vehicle, the tilt of the aforementioned mounting surface will be greater than the tilt when the robot's hand is outside the machine tool and the arm is not overhanging from the automated guided vehicle, or if it is overhanging, it is only a small amount.

[0010] Therefore, even if a visual target, which is a calibration marker, is placed on the outer surface of the machine tool, as in the conventional position correction method described above, and the robot's position correction amount (attitude correction amount) is obtained while the robot is outside the machine tool, the obtained position correction amount cannot be used to accurately correct the robot's attitude for workpiece attachment and detachment operations performed when the robot's hand is inside the machine tool.

[0011] Furthermore, if the robot's posture cannot be accurately corrected when attaching or detaching the workpiece, the robot hand cannot be accurately positioned relative to the chuck. For example, if the chuck is a collet chuck or similar, where the movement (stroke) of the gripping part is very small, that is, the clearance between the workpiece and the chuck is very small, there is a possibility that the workpiece cannot be securely gripped by the chuck.

[0012] Furthermore, if the loading and unloading of workpieces cannot be performed reliably, the operating rate of the system will decrease. [Means for solving the problem]

[0013] Therefore, this disclosure provides the systems, transport devices, image processing methods, image processing devices, etc., described in the claims. [Effects of the Invention]

[0014] According to this disclosure, the robot's operating posture can be corrected with greater accuracy by correcting the robot's operating posture based on an image containing an identification figure. Also, according to the present disclosure, by correcting the robot's operating posture based on an image showing the internal structure of the machine tool, the operating posture of the robot can be corrected with higher accuracy.

Brief Description of the Drawings

[0015] [Figure 1] It is a plan view showing a schematic configuration of a system according to an embodiment of the present invention. [Figure 2] It is a block diagram showing the configuration of the system according to the present embodiment. [Figure 3] It is a perspective view showing an automated guided vehicle and a robot according to the present embodiment. [Figure 4] It is an explanatory diagram for explaining the imaging posture of the robot according to the present embodiment. [Figure 5] It is an explanatory diagram showing the relationship between each coordinate system set in the present embodiment. [Figure 6] It is an explanatory diagram showing an identification graphic according to the present embodiment. <( [Figure 7] It is an explanatory diagram showing the working posture of the hand corrected based on the image of the identification graphic captured during actual operation when there is no error in the imaging posture of the camera during teaching operation and actual operation. [Figure 8] It is an explanatory diagram showing an image of the identification graphic captured by the camera during teaching operation. [Figure 9] It is an explanatory diagram showing a case where there is an error in the imaging posture of the camera during teaching operation and actual operation. [Figure 10] It is an explanatory diagram showing images of the identification graphic captured in respective postures when there is an error in the imaging posture of the camera during teaching operation and actual operation. [Figure 11] It is an explanatory diagram showing the working posture of the hand corrected based on the image of the identification graphic captured during actual operation when there is an error in the imaging posture of the camera during teaching operation and actual operation. [Figure 12]It is an explanatory diagram showing an aspect of correcting the imaging posture of a camera during actual operation so as to approach the imaging posture during teaching operation when there is an error in the imaging posture of the camera during teaching operation and actual operation. [Figure 13] It is an explanatory diagram showing an aspect of correcting the operating posture of a hand based on an image of an identification figure captured in a state where the imaging posture of a camera during actual operation is corrected. [Figure 14] In the present embodiment, it is a flowchart showing the operation modes of an automated guided vehicle and a robot executed by automatic driving control. [Figure 15] It is an explanatory diagram showing a modification example in which an identification figure is arranged in a machine tool. [Figure 16] It is an explanatory diagram showing another modification example in which an identification figure is arranged in a machine tool.

Modes for Carrying Out the Invention

[0016] Hereinafter, specific embodiments will be described with reference to the drawings.

[0017] [Configuration of the System According to the Present Embodiment] As shown in FIGS. 1 and 2, the system 1 according to the present embodiment includes a machine tool 10, a material stocker 20 and a product stocker 21 as peripheral devices, an automated guided vehicle 35, a robot 25 mounted on the automated guided vehicle 3�, a camera 31 attached to the robot 25, and a control device 40 that controls the robot 25 and the automated guided vehicle 35. Note that a robot-mounted transfer device is configured from the robot 25, the automated guided vehicle 35, and the control device 40.

[0018] As shown in Figure 4, the machine tool 10 is a vertical NC (numerical control) lathe equipped with a spindle 11 to which a chuck 12 for gripping a workpiece W(W') is attached, and the spindle 11 is positioned vertically, enabling turning of the workpiece W(W'). A tool presetter 13 equipped with a contact element 14 and a support bar 15 for supporting it is provided near the spindle 11. The support bar 15 is positioned to move back and forth along the axis of the spindle 11 in relation to the machining area, and a ceramic display plate 16 (this is just one example, and is not limited to ceramic) is provided on the end face of the display plate 16, on which the identification figure shown in Figure 6 is drawn. The display plate 16 is positioned on a horizontal plane. Thus, in this embodiment, the identification figure is placed inside the machine tool 10, and is particularly preferred to be placed inside the machining area.

[0019] Furthermore, Figure 4 shows the state in which the support bar 15 and contact element 14 have advanced into the processing area. However, when the support bar 15 and contact element 14 retract, and the contact element 14 and display plate 16 are stored in the storage area, the shutter 17 closes, isolating the contact element 14 and display plate 16 from the processing area.

[0020] The identification figure in this example has a matrix structure in which multiple square pixels are arranged in two dimensions, and each pixel is displayed in either white or black. In Figure 6, the black pixels are marked with diagonal lines. Such identification figures are sometimes called AR markers or AprilTag. If the identification figure is small, a lens may be placed on the identification figure so that an enlarged image is captured by the camera 31 described later.

[0021] The material stocker 20 is located to the left of the machine tool 10 in Figure 1 and is a device for stocking multiple materials (workpieces W before processing) to be processed by the machine tool 10. The product stocker 21 is located to the right of the machine tool 10 in Figure 1 and is a device for stocking multiple products or semi-finished products (processed workpieces W') processed by the machine tool 10.

[0022] As shown in Figure 1, the automated guided vehicle 35 has the robot 25 mounted on its upper surface, which is the mounting surface 36, and is also equipped with a portable control panel 37 for the operator. The control panel 37 includes an input / output unit for inputting and outputting data, an operation unit for manually operating the automated guided vehicle 35 and the robot 25, and a display capable of showing screen information.

[0023] Furthermore, the automated guided vehicle 35 is equipped with a sensor (for example, a distance measuring sensor using laser light) that can recognize its own position within the factory, and is configured to travel tracklessly within the factory, including the area where the machine tool 10, material stocker 20, and product stocker 21 are located, under the control of the control device 40, and in this example, it passes through each of the work positions set for the machine tool 10, material stocker 20, and product stocker 21, respectively.

[0024] As shown in Figures 1 and 3, the robot 25 of this embodiment is a multi-joint robot equipped with three arms: a first arm 26, a second arm 27, and a third arm 28. A hand 29 is attached to the tip of the third arm 28 as an end effector, and a camera 31 is attached via a support bar 30. However, applicable robots are not limited to this configuration. A robot only needs to have (i) a camera, (ii) a hand for gripping a workpiece or tool, (iii) a second arm that movably connects the hand, and (iv) a first arm that movably connects the second arm. In comparison to the robot 25 of this embodiment, the hand corresponds to the hand 29, the second arm corresponds to the joint that rotatably (movably) connects to the second arm 27, and the first arm corresponds to the joint that rotatably (movably) connects to the first arm 26. It may also be understood that the joint portion that rotatably and reciprocally (movably) connects the third arm 28 of the robot in this embodiment corresponds to the second arm portion. In other words, although there are three arms in this embodiment, at least two arms are sufficient.

[0025] As shown in Figure 2, the control device 40 of this embodiment consists of an operation program storage unit 41, a movement position storage unit 42, an operation posture storage unit 43, a map information storage unit 44, an identification graphic image storage unit 45, a manual operation control unit 46, an automatic operation control unit 47, a map information generation unit 48, a position recognition unit 49, and an input / output interface 50. The control device 40 is connected to the machine tool 10, material stocker 20, product stocker 21, robot 25, camera 31, automated guided vehicle 35, and control panel 37 via this input / output interface 50. Note that the control device 40 is not limited to this configuration. The control device 40 only needs to have a function unit for controlling the robot 25, and other function units may be provided by other devices.

[0026] The control device 40 is composed of a computer including a CPU, RAM, ROM, etc., and the manual driving control unit 46, automatic driving control unit 47, map information generation unit 48, position recognition unit 49, and input / output interface 50 have their functions realized by computer programs and execute the processes described later. In addition, the operation program storage unit 41, movement position storage unit 42, operation posture storage unit 43, map information storage unit 44, and identification graphic image storage unit 45 are composed of appropriate storage media such as RAM. In this example, the control device 40 is attached to the automated guided vehicle 35 and is connected to the machine tool 10, material stocker 20, and product stocker 21 by appropriate communication means, and is also connected to the robot 25, camera 31, automated guided vehicle 35, and control panel 37 by wired or wireless means. However, it is not limited to this configuration, and the control device 40 may be installed at an appropriate location other than the automated guided vehicle 35. In this case, the control device 40 is connected to each part by appropriate communication means.

[0027] The manual operation control unit 46 is a functional unit that operates the automated guided vehicle 35, robot 25, and camera 31 according to operation signals input by the operator from the control panel 37. In other words, the operator can perform manual operations of the automated guided vehicle 35, robot 25, and camera 31 using the control panel 37 under the control of the manual operation control unit 46.

[0028] The operation program storage unit 41 is a functional unit that stores an automatic driving program for automatically operating the automated guided vehicle 35 and the robot 25 during production, and a map generation program for operating the automated guided vehicle 35 when generating map information within the factory, which will be described later. The automatic driving program and the map generation program are input, for example, from an input / output unit provided on the control panel 37 and stored in the operation program storage unit 41.

[0029] Furthermore, this autonomous driving program includes command codes for the target location, speed, and orientation of the automated guided vehicle (AGV) 35, as well as command codes for the sequential actions of the robot 25 and command codes for the operation of the camera 31. The map generation program also includes command codes for the AGV 35 to travel tracklessly throughout the factory so that the map information generation unit 48 can generate map information.

[0030] The map information storage unit 44 is a functional unit that stores map information including information on the placement of machinery, equipment, devices, etc. (devices, etc.) located within the factory where the automated guided vehicle 35 travels, and this map information is generated by the map information generation unit 48.

[0031] The map information generation unit 48, under the control of the automatic driving control unit 47 of the control device 40 (described in more detail later), operates the automated guided vehicle 35 according to the map generation program stored in the operation program storage unit 41. At this time, it acquires spatial information within the factory from distance data detected by the sensors and recognizes the planar shapes of equipment and other devices installed within the factory. For example, based on the planar shapes of equipment and other devices registered in advance, it recognizes the positions, planar shapes, etc. (arrangement information) of specific equipment installed within the factory, in this example, the machine tool 10, the material stocker 20, and the product stocker 21. The map information generation unit 48 then stores the obtained spatial information and arrangement information of equipment and other devices in the map information storage unit 44 as map information of the factory.

[0032] The position recognition unit 49 is a functional unit that recognizes the position of the automated guided vehicle 35 within the factory based on distance data detected by the sensor and map information of the factory stored in the map information storage unit 44. Based on the position of the automated guided vehicle 35 recognized by the position recognition unit 49, the operation of the automated guided vehicle 35 is controlled by the automatic driving control unit 47.

[0033] The aforementioned movement position storage unit 42 is a functional unit that stores movement positions that serve as specific target positions to which the automated guided vehicle 35 moves, and which correspond to specific movement positions corresponding to command codes in the operation program. These movement positions include the work positions set for the machine tool 10, material stocker 20, and product stocker 21 as described above. These movement positions are set, for example, by manually operating the automated guided vehicle 35 using the operation panel 37 under the control of the manual operation control unit 46 to move it to each target position, and then storing the position data recognized by the position recognition unit 49 in the movement position storage unit 42. This operation is called a teaching operation.

[0034] The aforementioned operating posture storage unit 43 is a functional unit that stores data relating to the operating posture, which is the posture (operating posture) of the robot 25 that changes sequentially as the robot 25 operates in a predetermined order, and which corresponds to the command code in the operation program. This data relating to the operating posture is the rotation angle data of each joint (motor) of the robot 25 in each of the target postures when the robot 25 is manually operated by teaching operations using the operation panel 37 under the control of the manual operation control unit 46 and made to assume each target posture. This rotation angle data is stored in the operating posture storage unit 43 as data relating to the operating posture.

[0035] The specific operating postures of the robot 25 are set in the material stocker 20, the machine tool 10, and the product stocker 21, respectively. For example, in the material stocker 20, the operating postures are set as follows: the operating posture when starting work in the material stocker 20 (starting retrieval posture), each operating posture (each retrieval posture) for grasping the unprocessed workpiece W stored in the material stocker 20 with the hand 29 and retrieving it from the material stocker 20, and the posture when retrieval is completed (the retrieval completion posture, which in this example is the same as the starting retrieval posture).

[0036] Furthermore, the machine tool 10 is configured to have a workpiece removal position for removing the processed workpiece W' from the machine tool 10, and a workpiece mounting position for attaching the workpiece W before processing to the machine tool 10.

[0037] Specifically, in the workpiece removal operation posture, for example, the following postures are set: the work start posture before entering the machine tool 10; the posture in which the hand 29 and camera 31 enter the machining area of ​​the machine tool 10 and the camera 31 takes an image of the identification figure provided on the support bar 15 (imaging posture) (see Figure 4); the posture in which the hand 29 is facing the machined workpiece W' held in the chuck 12 of the machine tool 10 (removal preparation posture); the posture in which the hand 29 is moved towards the chuck 12 and the machined workpiece W' held in the chuck 12 is grasped by the hand 29 (grasping posture); the posture in which the hand 29 is separated from the chuck 12 and the machined workpiece W' is removed from the chuck 12 (removal posture); and the posture in which the hand 29 and camera 31 have left the machine tool 10 (work completion posture).

[0038] Furthermore, the workpiece mounting operation postures include, for example, the starting posture before entering the machine tool 10, the posture in which the hand 29 and camera 31 enter the machining area of ​​the machine tool 10 and the camera 31 captures an image of the identification figure provided on the support bar 15 (imaging posture) (see Figure 4), the posture in which the workpiece W before machining, held by the hand 29, is facing the chuck 12 of the machine tool 10 (mounting preparation posture), the posture in which the hand 29 is moved toward the chuck 12 so that the workpiece W before machining can be gripped by the chuck 12 (mounting posture), the posture in which the hand 29 is separated from the chuck 12 (separation posture), and the posture in which the hand 29 and camera 31 are removed from the machine tool 10 (work completion posture).

[0039] In the product stocker 21, the following are set as storage operation postures: the starting posture when starting work in the product stocker 21 (storage start posture), each working posture (storage posture) for storing the processed workpiece W' held by the hand 29 into the product stocker 21, and the posture when storage is completed (storage completion posture, which in this example is the same posture as the storage start posture).

[0040] The identification figure image storage unit 45 is a functional unit that stores images obtained by the camera 31 of the identification figure provided on the support bar 15 of the tool presetter 13 when the automated guided vehicle 35 is in the set working position relative to the machine tool 10 and the robot 25 is in the imaging posture, during teaching operations and automatic operation. The image of the identification figure captured during teaching operations is stored in the identification figure image storage unit 45 as a reference image. In addition, the imaging posture position and mounting posture position in the figure coordinate system are stored in the storage unit as appropriate.

[0041] The automatic driving control unit 47 is a functional unit that uses either the automatic driving program or the map generation program stored in the operation program storage unit 41 to operate the automated guided vehicle 35, the robot 25, and the camera 31 according to the program. In doing so, the data stored in the movement position storage unit 42 and the operation posture storage unit 43 is used as needed.

[0042] [Unmanned automated production using the system of this embodiment] According to the system 1 of this embodiment, unmanned automated production is performed under the control of the automatic operation control unit 47 as follows.

[0043] In other words, the automatic driving control unit 47 executes the automatic driving program stored in the operation program storage unit 41, and for example, the automated guided vehicle 35 and the robot 25 operate as follows.

[0044] First, the automated guided vehicle 35 moves to the work position set for the machine tool 10, and the robot 25 assumes the starting position for the workpiece removal operation described above. At this time, the machine tool 10 has completed the predetermined machining and opened its door cover to allow the robot 25 to enter the machining area, and has also received a command from the automatic operation control unit 47 to extend the support bar 15 of the tool presetter 13 into the machining area.

[0045] Next, the automatic operation control unit 47 instructs the robot 25 to perform the subsequent workpiece removal operation, grasping the processed workpiece W' held in the chuck 12 of the machine tool 10 with the hand 29 and removing it from the machine tool 10. At this time, the automatic operation control unit 47 has the robot 25 assume an imaging posture, captures the identification figure with the camera 31, and corrects the subsequent removal preparation posture, gripping posture, and removal posture based on the obtained image. As described above, because the automated guided vehicle 35 is a self-propelled vehicle using wheels, its positioning accuracy when stopping at the work position is low. For this reason, during actual operation, it is necessary to correct each operation posture that has been set in advance by teaching operations. The details of this correction will be described later. After the robot 25 assumes the gripping posture, the automatic operation control unit 47 sends a chuck open command to the machine tool 10, thereby opening the chuck 12.

[0046] Next, the automated driving control unit 47 moves the automated guided vehicle 35 to the work position set relative to the product stocker 21, and instructs the robot 25 to sequentially assume the storage start position when starting work in the product stocker 21, the storage positions for storing the processed workpiece held by the hand 29 into the product stocker 21, and the storage completion position when storage is complete, thereby storing the processed workpiece held by the hand 29 into the product stocker 21.

[0047] Next, the automatic driving control unit 47 moves the automated guided vehicle 35 to the work position set relative to the material stocker 20, and instructs the robot 25 to sequentially assume the retrieval start position when starting work in the material stocker 20, the retrieval positions for grasping the unprocessed workpiece stored in the material stocker 20 with the hand 29 and removing it from the material stocker 20, and the retrieval completion position when the retrieval is completed, thereby causing the hand 29 to grasp the unprocessed workpiece.

[0048] Next, the automatic operation control unit 47 moves the automated guided vehicle 35 to the set work position relative to the machine tool 10 again, then causes the robot 25 to perform the workpiece mounting operation described above, attaching the pre-processing workpiece W, which is held by the hand 29, to the chuck 12 of the machine tool 10, and then exits the machine. At this time, the automatic operation control unit 47 corrects the subsequent mounting preparation posture, mounting posture, and separation posture based on the image of the identification figure captured by the camera 31 while the robot 25 is in the imaging posture. After this, the automatic operation control unit 47 sends a machining start command to the machine tool 10, causing the machine tool 10 to perform the machining operation. After the robot 25 is in the mounting posture, the automatic operation control unit 47 sends a chuck closing command to the machine tool 10, causing the chuck 12 to close and the pre-processing workpiece W to be held by the chuck 12.

[0049] In System 1 of this example, unmanned automated production is continuously executed by repeating the above steps.

[0050] [Correction of movement posture] The following describes the posture correction methods described above with respect to the operating posture of the robot 25 performed on the machine tool 10. <Basic Concepts in Correction> First, the basic concept of posture correction in this example will be explained. As shown in Figure 5, a robot coordinate system, camera coordinate system, graphic coordinate system, and goal coordinate system are set for the robot 25, camera 31, identification graphic, and target operating position (target operating position), respectively, determined by the orthogonal three axes x, y, and z. The origin of the robot coordinate system is arbitrarily set in the control space of the automatic driving control unit 47 (in this example, it is set at the base of the robot arm), the origin of the camera coordinate system is set, for example, at the center of the image sensor arranged on the two-dimensional plane of the camera 31, the origin of the graphic coordinate system is set at the center of the identification graphic, and the origin of the goal coordinate system is set at the target operating position. Note that in Figure 5, for convenience, only the robot coordinate system is labeled. Also in Figure 5, the target operating position of the robot 25 is the position just before the chuck 12 grips the workpiece W, and at the position just before the chuck 12, the axis of the workpiece W is coaxial with the axis of the chuck 12, which is the position of the hand 29. After this, by moving the hand 29 in the negative z-axis direction, the chuck 12 becomes able to grip the workpiece W.

[0051] Then, based on the control space information of the automatic driving control unit 47, and the mechanical design data (e.g., CAD data) of the robot 25 and camera 31, the coordinate values ​​(x, y, z) of the camera 31 in the robot coordinate system and the rotation angle values ​​(rx, ry, rz) around each coordinate axis are obtained, and a transformation matrix from the robot coordinate system to the camera coordinate system is calculated. Define it as TIFF0007839739000001.tif37. Similarly, the transformation matrix from the camera coordinate system to the robot coordinate system is defined. TIFF0007839739000002.tif27 can also be calculated (obtained).

[0052] Furthermore, a transformation matrix from the camera coordinate system to the figure coordinate system is calculated based on the coordinate values ​​(x, y, z) of the figure in the camera coordinate system and the rotation angle values ​​(rx, ry, rz) around each coordinate axis, which can be obtained from the internal parameters of the camera 31, the homography matrix recognized from the captured image of the identified figure, the center coordinates, the corner coordinates, and the size of the identified figure. Define TIFF0007839739000003.tif37. Similarly, the transformation matrix from the geometric coordinate system to the camera coordinate system is Defined as TIFF0007839739000004.tif37.

[0053] <Processing of data obtained through teaching operations> The automatic driving control unit 47 first performs a transformation matrix from the graphic coordinate system to the camera coordinate system at the time of teaching, based on the image of the identification graphic (reference image) acquired during the teaching operation (hereinafter referred to as "teaching time") and stored in the identification graphic image storage unit 45. TIFF0007839739000005.tif312 and the transformation matrix from camera coordinate system to geometric coordinate system during teaching. Retrieve TIFF0007839739000006.tif312.

[0054] Next, the automatic driving control unit 47, for each operating posture acquired by the teaching operation and stored in the operating posture memory unit 43, performs a transformation matrix from the goal coordinate system of the hand 29 to the robot coordinate system at the time of teaching, according to a predetermined transformation formula set for control purposes. Calculate TIFF0007839739000007.tif310 and the calculated transformation matrix Based on TIFF0007839739000008.tif310, the transformation matrix from the goal coordinate system to the geometric coordinate system is given by the following equation 1. Calculate TIFF0007839739000009.tif35. (Equation 1) TIFF0007839739000010.tif329 Here, TIFF0007839739000011.tif310 is the transformation matrix from the robot coordinate system to the geometric coordinate system during teaching, and can be calculated by the following equation 2. (Equation 2) TIFF0007839739000012.tif337 Note: Transformation matrix TIFF0007839739000013.tif312 is calculated based on the image of the recognition shape captured during teaching, as described above. Also, the transformation matrix is... TIFF0007839739000014.tif312 is also obtained from the control space information of the automatic driving control unit 47, as well as from the mechanical design data (e.g., CAD data) of the robot 25 and camera 31, as described above.

[0055] <Correction of actual working posture> The automatic driving control unit 47 first creates a transformation matrix from the graphic coordinate system to the camera coordinate system during actual operation, based on the image of the identification graphic captured by the camera 31 during actual operation (hereinafter sometimes referred to as "currently"). TIFF0007839739000015.tif313, and the transformation matrix from the camera coordinate system to the geometric coordinate system during actual operation. Calculate (obtain) TIFF0007839739000016.tif313.

[0056] Next, the automatic driving control unit 47 generates a transformation matrix from the goal coordinate system to the robot coordinate system during actual operation. TIFF0007839739000017.tif312 is the transformation matrix from the goal coordinate system to the geometric coordinate system calculated by formula 1 above. Based on TIFF0007839739000018.tif35, the calculation is performed using the following formula 3. (Equation 3) TIFF0007839739000019.tif332 Here, TIFF0007839739000020.tif312 is the transformation matrix from the geometric coordinate system to the robot coordinate system during actual operation, and can be calculated by the following equation 4. (Equation 4) TIFF0007839739000021.tif341 Note: Transformation matrix TIFF0007839739000022.tif313 is a transformation matrix from the camera coordinate system during actual operation to the robot coordinate system during actual operation, and as described above, it is obtained from the control space information of the automatic driving control unit 47, as well as the mechanical design data (e.g., CAD data) of the robot 25 and camera 31. As mentioned above, TIFF0007839739000023.tif313 is obtained from the image of the identification figure captured during actual operation.

[0057] Then, the transformation matrix from the calculated (corrected) goal coordinate system to the robot coordinate system during actual operation is used. Based on TIFF0007839739000024.tif312, the target movement position of hand 29 in the robot coordinate system during actual operation. TIFF0007839739000025.tif337 and target operating angle TIFF0007839739000026.tif340 is calculated according to the following formulas 5, 6, and 7. (Equation 5) TIFF0007839739000027.tif539(Formula 6) TIFF0007839739000028.tif1025(Formula 7) TIFF0007839739000029.tif762 However, in formula 7 TIFF0007839739000030.tif22 TIFF0007839739000031.tif312, TIFF0007839739000032.tif22 TIFF0007839739000033.tif312, TIFF0007839739000034.tif22 The filename is TIFF0007839739000035.tif312. Hand 29 is the corrected target operating position. TIFF0007839739000036.tif337, for example, move to the position shown in Figure 7, and further target movement angle Rotate to TIFF0007839739000037.tif340. The example shown in Figure 7 shows that the hand 29 of the robot 25 is positioned without positional error at the target operating position, i.e., in front of the chuck 12, where the axis of the workpiece W is coaxial with the axis of the chuck 12.

[0058] <Correction of imaging posture during actual operation> Incidentally, when the camera 31 captures the identification figure during teaching, it is common sense to set the camera 31 so that the identification figure is located approximately in the center of the camera frame, taking into account positioning errors during actual operation, as shown in Figure 8. However, if the positioning accuracy of the automated guided vehicle 35 during actual operation is poor, the camera 31's position during actual operation (solid line) may differ significantly from its position during teaching (dotted line), as shown in Figure 9. If the camera 31's position is significantly displaced, the identification figure may shift to the edge of the camera frame during actual operation, or the camera 31 may move closer to or further away from the identification figure, as shown in Figure 10.

[0059] When this happens, the image of the identification figure captured by camera 31 becomes blurry or its size is altered, and the above transformation matrix obtained from this identification figure is lost. The file TIFF0007839739000038.tif313 contains an error, and the transformation matrix from the goal coordinate system to the robot coordinate system during actual operation is calculated based on this error. TIFF0007839739000039.tif312, Target movement position of hand 29 in the robot coordinate system during actual operation TIFF0007839739000040.tif337 and target operating angle Errors will also be present in TIFF0007839739000041.tif340. As a result, as shown in Figure 11, it becomes impossible to accurately position the robot 25's hand 29 to the target operating position. In the example shown in Figure 11, the robot 25's hand 29 is positioned at a location deviated from the target operating position, that is, in front of the chuck 12, and the axis of the workpiece W is positioned away from the axis of the chuck 12. After this, if the hand 29 is moved in the negative z-axis direction, the workpiece W will collide with the chuck 12, and the chuck 12 will not be able to grip the workpiece W.

[0060] Therefore, in this embodiment, during automatic operation by the automatic driving control unit 47, the imaging posture during actual operation is compared with the imaging posture during teaching. If the error in the imaging posture is not within a predetermined acceptable range (threshold), a process is executed to correct the imaging posture during actual operation so that the error falls within the acceptable range. If the error does not fall within the acceptable range after one correction, the imaging posture correction process is repeated until the error falls within the acceptable range.

[0061] <Confirmation of imaging posture> The automatic driving control unit 47 converts the image of the identification figure captured during teaching into a transformation matrix from the figure coordinate system to the camera coordinate system used during teaching. Based on TIFF0007839739000042.tif312, the position and angle of the figure in the camera coordinate system during teaching. Obtain TIFF0007839739000043.tif377. Also, obtain the transformation matrix from the graphic coordinate system to the camera coordinate system during actual operation, based on the image of the identification graphic captured during actual operation. Retrieve TIFF0007839739000044.tif313 and determine the shape's position and angle in the camera coordinate system during actual operation. First, calculate TIFF0007839739000045.tif871. Next, calculate the difference values ​​(Δx, Δy, Δz, Δrx, Δry, Δrz). If any of these difference values ​​are outside the predetermined tolerance range, i.e., if all of the following conditions are not met, correct the imaging orientation. -0.05mm ≤ Δx ≤ 0.05mm -0.05mm ≤ Δy ≤ 0.05mm -0.05mm ≤ Δz ≤ 0.05mm -0.05°≦Δrx≦0.05° -0.05°≦Δry0.05° -0.05°≦Δrz≦0.05° The threshold values ​​for each difference are merely examples and are not limiting; they can be set empirically as appropriate to obtain images of the recognized shapes with good accuracy. In this example, the imaging orientation is corrected so that all difference values ​​fall within the acceptable range, but this is not limiting; the imaging orientation may also be corrected so that the difference value of either the coordinate values ​​(x,y,z) or the rotation angle values ​​(rx,ry,rz) falls within the acceptable range.

[0062] <Correction of imaging orientation> The automatic driving control unit 47 generates a transformation matrix from the camera coordinate system during teaching to the robot coordinate system during actual operation. TIFF0007839739000046.tif312 is the transformation matrix from the camera coordinate system to the geometric coordinate system during teaching. Based on TIFF0007839739000047.tif312, the following formula 8 is used to calculate it. (Equation 8) TIFF0007839739000048.tif338 Here, TIFF0007839739000049.tif312 is a transformation matrix from the geometric coordinate system to the robot coordinate system during actual operation, and can be calculated using equation 4 described above.

[0063] The automatic driving control unit 47 then converts the position of the camera 31 from the camera coordinate system during teaching to the robot coordinate system during actual operation using a transformation matrix. Camera position calculated based on TIFF0007839739000050.tif312 TIFF0007839739000051.tif338 and angle The imaging posture of the robot 25 is corrected by moving to TIFF0007839739000052.tif340 (see Figure 12. In Figure 12, the position (posture) of the camera 31 is corrected from the position (posture) shown by the dashed line to the position (posture) shown by the solid line). Note that this corrected camera position is the camera position during teaching in the robot coordinate system during actual operation. Under normal circumstances, this correction makes the position (posture) of the camera 31 during actual operation the same as the position (posture) during teaching. However, because the posture of the robot 25 is changed by the correction, depending on the behavior of the automated guided vehicle 35, the difference between the corrected camera position and the camera position during teaching may not fall within the acceptable range. In this case, this correction process is repeated until the error between the imaging posture during actual operation and the imaging posture during teaching falls within the acceptable range.

[0064] In this way, if the camera position during actual operation differs from the camera position during teaching, the automatic driving control unit 47 corrects the imaging posture of the robot 25 so that the camera position during actual operation is approximately the same as the camera position during teaching, then captures the identification figure with the camera 31, and based on the obtained image, corrects the target operating position of the hand 29 in each subsequent operating posture according to the formula 3, and the hand 29 moves to the corrected target operating position TIFF0007839739000053.tif337 and target operating angle The robot 25 is controlled to move to TIFF0007839739000054.tif340. As a result, the hand 29 moves to the target operating position set during teaching, for example, as shown in Figure 13. Figure 13 shows that the hand 29 of the robot 25 is positioned at the target operating position without any positional error, and that the axis of the workpiece W is positioned coaxial with the axis of the chuck 12 in front of the chuck 12.

[0065] [Summary explanation regarding the operation control of robot-mounted transport devices for machine tools] Next, based on the above, a general explanation of the operation control of the robot-mounted transfer device for the machine tool 10 will be given based on Figure 14.

[0066] When a robot-mounted transport device performs work on the machine tool 10, the automatic operation control unit 47 controls the operation of the robot-mounted transport device as follows.

[0067] Specifically, the automatic driving control unit 47 first moves the automated guided vehicle 35 to the work position set for the machine tool and then stops it (step S1). Next, the automatic driving control unit 47 has the robot 25 assume a work start posture (step S2), then assume an imaging posture (step S3), and in this state, the camera 31 captures an image of the identification figure (step S4). Then, based on the obtained current identification figure and the image of the identification figure captured during teaching, the camera 31 determines the position and angle of the figure in the current camera coordinate system. TIFF0007839739000055.tif873, and the position and angle of the figure in the camera coordinate system during teaching. The file TIFF0007839739000056.tif377 is obtained, and then the difference values ​​(Δx, Δy, Δz, Δrx, Δry, Δrz) are calculated to verify (determine) whether each difference value falls within the aforementioned acceptable range (Step S5).

[0068] Then, in step S5, if it is determined that each difference value is not within the above acceptable range, steps S4 and S5 are repeated while correcting the imaging posture (step S6) until each difference value falls within the above acceptable range.

[0069] On the other hand, if all difference values ​​are within the acceptable range, the automatic driving control unit 47, according to the above formula 3, determines the target operating position and angle corresponding to the operating position corrected for each operating position during teaching. TIFF0007839739000057.tif378 is calculated (step S7), and the robot 25 is given the calculated target motion position and angle. The system is made to sequentially assume each operating position corresponding to TIFF0007839739000058.tif378 (step S8), then assume the work completion position (step S9), and finally terminate the process.

[0070] As described in detail above, according to the system 1 of this embodiment, when correcting the operating posture of the robot 25 based on the image of the identification figure captured by the camera 31, the camera 31's imaging posture during actual operation is corrected so that the error between the camera 31's imaging posture during teaching and its imaging posture during actual operation falls within an acceptable range. As a result, the identification image captured during actual operation can be made clear and accurate, and the correction of the robot 25's operating posture using this image can be performed with greater accuracy.

[0071] Furthermore, in System 1 of this example, the robot 25's working posture is corrected using identification figures placed within the machining area of ​​the machine tool 10 where the robot 25 actually works. This allows for accurate correction of the working posture, enabling the robot 25 to perform tasks with high precision, even those requiring high operational accuracy.

[0072] Furthermore, by having the robot 25 perform tasks with high precision, the system 1 operates at a high utilization rate without causing unnecessary interruptions. As a result, the system 1 makes it possible to achieve highly reliable and highly efficient unmanned operation.

[0073] Furthermore, in this example, when the machine tool 10 is performing machining, an identification figure is provided on the support bar 15 of the tool presetter 13, which is stored outside the machining area. This prevents the identification figure from being contaminated by chips and other debris generated during machining, and as a result, the above correction can be performed with high accuracy.

[0074] Furthermore, in this example, the identification figure has a matrix structure in which multiple pixels are arranged in two dimensions, so that the motion posture can be corrected with high precision.

[0075] Although one embodiment of the present invention has been described above, the specific embodiments that the present invention may adopt are not limited thereto.

[0076] For example, in the embodiment described above, the identification figure was a matrix structure in which multiple pixels are arranged in two dimensions, but it is not limited to this, and various figures can be used as long as they can correct the posture of the robot 25 from the captured image.

[0077] Furthermore, in this embodiment, the operating posture of the robot 25 during actual operation is corrected based on the captured image of the identification figure without calculating the error amount between the teaching posture and the actual operation posture. However, this is not the only method, and the operating posture of the robot 25 during actual operation can be corrected using any other possible method. For example, the error amount between the imaging posture during teaching and the imaging posture during actual operation when the robot 25 captures the identification figure may be calculated, and other operating postures of the robot 25 may be corrected based on this error amount.

[0078] Furthermore, in this embodiment, the operating posture of the robot 25 is corrected for errors in its three-dimensional space, but it is not limited to this, and errors in a plane defined by two specific orthogonal axes may also be corrected. For example, the identification figure may be placed horizontally, and positional errors in the first and second axis directions within the plane containing the identification figure, as well as rotational errors around an axis orthogonal to the plane, may be corrected.

[0079] Furthermore, while the above embodiments illustrate configurations using an automated guided vehicle 35, the system is not limited thereto. A transport device that can be pushed by a person, like a general trolley, may also be used. Alternatively, a robot 25 may be mounted on this transport device, and the transport device may be manually transported to the working position of the machine tool 10, with the robot 25 performing the loading and unloading of workpieces onto the machine tool 10.

[0080] Furthermore, in the above example, the display board 16, i.e., the identification figure, was placed horizontally within the machine tool 10, but this is not the only option; it may also be placed parallel to the vertical plane.

[0081] Furthermore, although vertical lathes were used as examples of machine tools in the embodiments described above, the invention is not limited to these, and any conventionally known machine tools can be applied, including horizontal lathes, vertical and horizontal machining centers, and multi-tasking machines equipped with a tool spindle and a workpiece spindle.

[0082] For example, in the case of a horizontal lathe 100 equipped with a tool spindle 105 for rotating a tool, as shown in Figure 15, the display plate 16 can be supported horizontally by a holder 106, and this holder 106 can be mounted on the tool spindle 105, as shown in Figure 15. Alternatively, the display plate 112 can be supported vertically by a holder 111, as shown in Figure 16. In these cases, when machining is performed by the lathes 100 and 110, the holders 106 and 111 are stored in a tool magazine, which is a tool storage section, and when the robot 25 performs the work, the holders 106 and 111 are taken out of the tool magazine and mounted on the tool spindle 105. In Figures 15 and 16, reference numeral 101 denotes the first spindle and reference numeral 103 denotes the second spindle, and these are arranged coaxially and opposite to each other. Furthermore, reference numeral 102 denotes a first chuck mounted on the first spindle 101, and reference numeral 104 denotes a second chuck mounted on the second spindle 103. Also, reference numeral 107 denotes a tool post, reference numeral 108 denotes a turret provided on the tool post 107, and reference numeral 109 denotes a support jig for supporting the workpiece W'', which is attached to the outer surface of the turret 108.

[0083] Furthermore, in the above embodiment, the x and y axes were set in the horizontal plane and the z axis in the vertical direction in the robot coordinate system, camera coordinate system, geometric coordinate system, and goal coordinate system. However, this is not limited to this, and the direction of the coordinate axes can be set arbitrarily.

[0084] Furthermore, while the above embodiment mainly described an example in which the workpiece W(W') is attached and detached by the robot 25, it is not limited to this, and the objects handled by the robot 25 include not only the workpiece W(W') but also tools, ATC cameras, measuring instruments, and other items that can be attached and detached from the machine tool 10.

[0085] The following variations are also possible. (modified version) In previous embodiments, we described an example in which an image containing an identification figure is captured, the position of the camera that captured the image is calculated from the position of the identification figure in the image, and the camera is controlled to move to a preset position. However, we are not limited to this form.

[0086] In this embodiment, without using identification figures, the contour of the internal shape of a machine tool captured in a 2D image is detected from the captured image and compared with CAD data of a pre-set shape. The ratio of matching points between the contour data extracted from the image and the 2D shape data generated based on the CAD data is evaluated, and shapes with a high degree of agreement (many points matching the pre-set shape) are identified, and the camera position is calculated from the position of that shape. With this configuration, since identification figures are not used, the shape of the internal structure of the machine tool can be used as a reference. For example, internal structures include chucks, tools, spindles, turrets, tool presetters, tables, and pallets. Of course, the internal structure may also be an identification figure.

[0087] Three-dimensional recognition based on the contour (shape) of the internal structure may be determined by evaluating the number of matching points between edges extracted from an image and contours obtained by projecting a three-dimensional model created with 3D CAD or similar software onto a two-dimensional image. Three-dimensional recognition based on a three-dimensional point cloud may be evaluated and determined based on the number of matching points between a three-dimensional point cloud measured by a predetermined method and a three-dimensional model.

[0088] And the following image processing can be performed. An image processing method for processing an image captured by a camera of a robot-mounted transport device, comprising: (i) a camera for capturing images; (ii) a robot to which the camera is attached and which has an action part that acts on an object; (iii) a mobile device on which the robot is mounted and which is configured to be movable; and (iv) a control device for controlling the position of the action part, wherein A first calculation step involves analyzing a first image captured by the camera showing the internal structure inside the machine tool, and calculating the current position of the camera from the analysis of the internal structure shown in the first image. A first control step involves controlling the robot's movement in order to move the camera from its current position to a pre-set camera position. After moving the camera, the camera captures an image of the internal structure inside the machine tool, the internal structure in the captured second image is analyzed, and the current position of the working part is calculated in a second calculation step. The image processing method comprises a second control step of controlling the movement of a robot in order to move the working part from its current position to a preset position of the object or a transport position for transporting the object.

[0089] Furthermore, we can also provide the following image processing devices. An image processing device for processing images captured by a camera of a robot-mounted transport device, comprising: (i) a camera for capturing images; (ii) a robot to which the camera is attached and which has an action part that acts on an object; (iii) a mobile device on which the robot is mounted and which is configured to be movable; and (iv) a control device for controlling the position of the action part, wherein A first calculation unit analyzes a first image captured by the camera showing the internal structure inside the machine tool, and calculates the current position of the camera from the analysis of the internal structure shown in the first image. A first control unit controls the movement of the robot in order to move the camera from its current position to a pre-set camera position, After moving the camera, the camera captures an image of the internal structure inside the machine tool, analyzes the internal structure in the captured second image, and calculates the current position of the working part. The image processing apparatus comprises a second control unit that controls the movement of a robot in order to move the working part from the current position of the working part to a preset position of the object or a transport position for transporting the object. Here, the camera's position is determined using the internal structure of the machine tool rather than an identification image. However, other configurations and controls that can be used when processing images containing identification shapes are captured can naturally be used in this embodiment as well.

[0090] Again, the above description of embodiments is illustrative and not restrictive in all respects. Modifications and alterations are readily possible for those skilled in the art. The scope of the present invention is indicated by the claims, not by the embodiments described above. Furthermore, the scope of the present invention includes modifications from the claims and embodiments equivalent thereto. [Explanation of Symbols]

[0091] 1 System 10 Machine tools 11 Spindle 12 Chuck 13 Tool Presets 14 Contactor 15 Support bar 16 Display board 20 Material Stocker 21 Product Stocker 25 Robots 29 Hand 31 Camera 35 Automated Guided Vehicles 37 Control panel 40 Control device 41 Operation Program Storage Unit 42 Movement position memory section 43 Motion posture memory section 44 Map Information Storage Unit 45 Identification graphic image storage unit 46 Manual Operation Control Unit 47. Automated Driving Control Unit 48 Map Information Generation Unit 49 Position recognition part 50 Input / Output Interfaces W Workpiece before processing W' Machined workpiece

Claims

1. An image processing method for processing an image captured by a camera of a robot-mounted transport device, comprising: (i) a camera for capturing images; (ii) a robot to which the camera is attached and which has an action part that acts on an object; (iii) a mobile device on which the robot is mounted and which is configured to be movable; and (iv) a control device for controlling the position of the action part, wherein The process involves analyzing a first image captured by the camera showing the internal structure inside the machine tool, and calculating the current coordinate position and rotation angle of the internal structure in a camera coordinate system with the camera as the origin, based on the analysis of the internal structure shown in the first image. The process involves controlling the robot's movement in order to move the camera to a preset camera position based on the current coordinate position and rotation angle of the internal structure, After moving the camera, the camera captures an image of the internal structure inside the machine tool, the internal structure in the captured second image is analyzed, and the current position of the working part is calculated. An image processing method comprising the step of controlling the movement of a robot in order to move the working part from the current position of the working part to a preset position of the object or a transport position for transporting the object.

2. The image processing method according to claim 1, wherein the internal structure is one of an identification figure, a chuck, a tool, a spindle, a turret, a tool presetter, a table, or a palette.

3. An image processing apparatus for processing images captured by a camera of a robot-mounted transport device, comprising: (i) a camera for capturing images; (ii) a robot to which the camera is attached and which has an action part that acts on an object; (iii) a mobile device on which the robot is mounted and which is configured to be movable; and (iv) a control device for controlling the position of the action part, wherein the apparatus processes images captured by the camera of the robot-mounted transport device, The first image captured by the camera showing the internal structure inside the machine tool is analyzed, and the current coordinate position and rotation angle of the internal structure in the camera coordinate system with the camera as the origin are calculated from the analysis of the internal structure shown in the first image. To move the camera to a preset camera position based on the current coordinate position and rotation angle of the internal structure, the robot's movement is controlled. After moving the camera, the camera is used to image the internal structure of the machine tool, the internal structure in the captured second image is analyzed, and the current position of the working part is calculated. An image processing device that controls the movement of a robot in order to move the working part from its current position to a preset position of the object or a transport position for transporting the object.

4. A robot having a camera for capturing images and an action part that acts on an object to be worked on, A transport device equipped with the aforementioned robot and configured to be movable to a set work position relative to the work object, The system includes a control device configured to sequentially cause the robot to assume a work start posture, an imaging posture for capturing images of posture correction identification figures provided on the work object using the camera, and one or more work postures for applying the action unit to the work object, according to an operation program that includes pre-set operation commands. The aforementioned work start posture, imaging posture, and work posture are predetermined by teaching the robot to the robot in the robot-mounted transport device, The aforementioned identification figure is formed on a predetermined plane and placed on the work object. The control device is During the teaching operation, with the robot in an imaging posture, the process involves obtaining the coordinate position and rotation angle of the identification figure in a camera coordinate system with the camera as the origin, based on the image of the identification figure captured by the camera during the teaching operation. When the robot is actually operated according to the operation program, the transport device moves to the work position and the robot transitions from the work start position to the imaging position, and the process of obtaining the coordinate position and rotation angle of the identification figure in the camera coordinate system with the camera as the origin, based on the image of the identification figure captured by the camera, A process to verify whether the difference between the coordinate position and rotation angle of the identification figure obtained during actual operation and the coordinate position and rotation angle of the identification figure during the teaching operation is within a predetermined tolerance range, If the difference value is not within the acceptable range, the system is configured to perform a process to correct the camera's imaging orientation relative to the identification figure so that the difference value is within the acceptable range. Furthermore, the control device operates the robot so that, when the difference value is within the allowable range, the robot assumes a corrected working posture, which is obtained by correcting the working posture during the teaching operation based on the image of the identification figure obtained immediately before.

5. A machine tool that performs a predetermined machining operation on a workpiece, A robot having a camera for capturing images and an action part that acts on the workpiece, and performing work on the machine tool, A transport device equipped with the aforementioned robot and configured to be movable to a set working position relative to the machine tool, The system includes a control device configured to sequentially cause the robot to assume a work start posture, an imaging posture in which the camera captures an identification figure for posture correction provided on the machine tool, and one or more working postures for applying the action part to the workpiece, according to an operation program that includes pre-set operation commands. The aforementioned work start posture, imaging posture, and work posture are a system that is set in advance by teaching the robot, The aforementioned identification figure is formed on a predetermined plane and is placed within the machining area of ​​the machine tool. The control device is During the teaching operation, with the robot in an imaging posture, the process involves obtaining the coordinate position and rotation angle of the identification figure in a camera coordinate system with the camera as the origin, based on the image of the identification figure captured by the camera during the teaching operation. When the robot is actually operated according to the operation program, the transport device moves to the work position and the robot transitions from the work start position to the imaging position, and the process of obtaining the coordinate position and rotation angle of the identification figure in the camera coordinate system with the camera as the origin, based on the image of the identification figure captured by the camera, A process to verify whether the difference between the coordinate position and rotation angle of the identification figure obtained during actual operation and the coordinate position and rotation angle of the identification figure during the teaching operation is within a predetermined tolerance range, If the difference value is not within the acceptable range, the system is configured to perform a process to correct the camera's imaging orientation relative to the identification figure so that the difference value is within the acceptable range. Furthermore, the control device is a system that operates the robot so that, when the difference value is within the allowable range, the robot adopts a corrected working posture, which is obtained by correcting the working posture during the teaching operation based on the image of the identification figure obtained immediately beforehand.

6. The system according to claim 5, wherein the control device is configured to repeatedly perform a process of correcting the imaging orientation of the camera with respect to the identification figure until the difference value falls within an acceptable range.

7. The transport device is an automated guided vehicle controlled by the control device, and is configured to pass through the work position set for the machine tool under the control of the control device, according to claim 5.

8. The system according to claim 5, wherein the identification figure has a matrix structure in which a plurality of pixels are arranged in two dimensions.

Images