Robot, autonomous traveling robot system, guiding method, control method, and program

Abstract

To provide a robot that is able to accurately move a dolly along a route while configuring the dolly with minimum equipment and cost, by giving the function of guiding to the robot mounted on the dolly.SOLUTION: A robot mounted on a dolly capable of autonomous traveling, includes: an arm capable of arbitrarily changing a position and a posture of a distal end thereof; a vision sensor attached to the distal end of the arm; a detection unit configured to detect an amount of deviation of the dolly from a designated route, based on a mark included in an image captured by the vision sensor; and a traveling instruction unit configured to instruct the dolly to correct a traveling position or a traveling posture of the dolly, based on the amount of deviation.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to a robot, an autonomous traveling robot system, a guidance method, a control method, and a program. [Background technology]

[0002] It has been considered to mount a robot on an automated guided vehicle (AGV) and automatically transport the robot to a work position (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 2680298 Summary of the Invention [Problem to be solved by the invention]

[0004] AGVs equipped with robots travel autonomously using guidance methods such as magnetic guidance and optical guidance. In general, conventional commercially available AGVs are equipped with sensors corresponding to the guidance method and various functions for autonomous travel control, which are highly functional and expensive. Therefore, for example, when a commercially available AGV is introduced in a factory, it may include functions that are not used in the factory (for example, automatic transport scheduling and map creation functions), and the equipment configuration and cost may be excessive for the purpose. In addition, commercially available AGVs are assumed to be operated in an environment where the running floor is flat and stable running is possible, and in an environment where the running floor is uneven, it may be difficult to run on the correct route.

[0005] Furthermore, if the AVG cannot move precisely to the correct working position, the relative positions of the robot and the workpiece will be misaligned, making it difficult for the robot to move the tool to the correct working point on the workpiece to perform the work, which will reduce the accuracy of the robot's work.

[0006] The object of the present disclosure is to provide a robot, an autonomous mobile robot system, a guidance method, a control method, and a program that can move accurately along a path by being guided by a robot mounted on a cart while configuring the cart with a minimum of equipment and cost. [Means for solving the problem]

[0007] According to one aspect of the present disclosure, the robot is mounted on an autonomously movable cart and includes an arm whose tip position and posture can be arbitrarily changed, a vision sensor attached to the tip of the arm, a detection unit that detects the amount of deviation of the cart from a specified route based on landmarks included in an image captured by the vision sensor, and a driving instruction unit that instructs the cart to correct the driving position or driving posture of the cart based on the amount of deviation.

[0008] According to one aspect of the present disclosure, the robot is a robot that automatically performs a specified task on a workpiece, and includes an arm whose tip position and posture can be arbitrarily changed, a vision sensor attached to the tip of the arm, a tool attached to the tip of the arm, a vision correction unit that corrects a user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor, and an arm control unit that controls the position and posture of the arm so that the tip point of the tool attached to the tip of the arm moves to a working point in the corrected user coordinate system.

[0009] According to one aspect of the present disclosure, an autonomous driving robot system is an autonomous driving robot system including a cart and the above-mentioned robot, wherein the driving instruction unit of the robot instructs the cart to drive autonomously when the image does not include the landmark, and the cart has a driving assistance sensor capable of detecting an amount of movement and a direction of travel, and a driving control device that, when the instruction for autonomous driving is received, causes the cart to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor.

[0010] According to one aspect of the present disclosure, a guidance method is a method for guiding an autonomously moving cart using a robot mounted on the cart and having an arm whose tip position and posture can be arbitrarily changed and a vision sensor attached to the tip of the arm, the method including the steps of detecting an amount of deviation of the cart from a specified route based on landmarks contained in an image captured by the vision sensor, and instructing the cart to correct its running position or posture based on the amount of deviation.

[0011] According to one aspect of the present disclosure, the program causes a robot mounted on an autonomously movable cart, having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor attached to the tip of the arm, to perform the following steps: detecting the amount of deviation of the cart from a specified route based on landmarks contained in an image captured by the vision sensor attached to the tip of the robot's arm; and instructing the cart to correct the running position or posture of the cart based on the amount of deviation.

[0012] According to one aspect of the present disclosure, a control method is a control method for a robot having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor and tool attached to the tip of the arm, and performing a specified task on a workpiece, the control method including the steps of: correcting a user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor; and controlling the position and posture of the arm so that the tip point of the tool moves to a working point in the corrected user coordinate system.

[0013] According to one aspect of the present disclosure, a program causes a robot having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor and tool attached to the tip of the arm, which performs a specified task on a workpiece, to execute the steps of: correcting a user coordinate system of a workpiece to match the actual position and posture of the workpiece based on an image captured by a vision sensor attached to the tip of the robot's arm; and controlling the position and posture of the arm so that the tip point of the tool attached to the tip of the arm moves to a working point in the corrected user coordinate system. Effect of the Invention

[0014] According to the above aspect, by using a robot mounted on a cart to guide the cart, the cart can be moved accurately along a route while being configured with a minimum of equipment and at a minimum of cost.

[0015] Furthermore, according to the above aspect, even if the robot deviates slightly from the working position, the tool can be accurately moved to the working point on the workpiece to perform the work. [Brief description of the drawings]

[0016] [Figure 1] 1 is a diagram showing an overall configuration of an autonomous traveling robot system according to a first embodiment. [Diagram 2] 1 is a block diagram showing the functional configuration of a robot control device and a travel control device according to a first embodiment. FIG. [Diagram 3] 5 is a flowchart showing an example of a process for guiding a cart by the robot according to the first embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of route information according to the first embodiment. [Diagram 5] FIG. 2 is a diagram for explaining a guidance process of a robot according to the first embodiment. [Figure 6] 5 is a flowchart showing an example of a travel control process for the bogie according to the first embodiment. [Figure 7] FIG. 11 is a diagram for explaining a guidance process of a robot according to the first modified example of the first embodiment. [Figure 8] FIG. 11 is a block diagram showing a functional configuration of a robot control device according to a second embodiment. [Figure 9] 13 is a flowchart showing an example of a vision correction process of a robot according to a second embodiment. [Figure 10] FIG. 11 is a first diagram for explaining the vision correction process of the robot according to the second embodiment. [Figure 11] FIG. 11 is a second diagram for explaining the vision correction process of the robot according to the second embodiment. [Figure 12] FIG. 11 is a third diagram for explaining the vision correction process of the robot according to the second embodiment. [Figure 13] FIG. 11 is a fourth diagram for explaining the vision correction process of the robot according to the second embodiment. [Figure 14] FIG. 5 is a fifth diagram for explaining the vision correction process of the robot according to the second embodiment. [Figure 15] FIG. 2 is a diagram illustrating an example of a hardware configuration of a robot control device and a travel control device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] <First embodiment> Hereinafter, the embodiment will be described in detail with reference to FIGS.

[0018] (Overall composition) FIG. 1 is a diagram showing the overall configuration of an autonomous mobile robot system according to the first embodiment. The autonomous mobile robot system 100 includes a robot 1, a cart 2, and an operation PC 3.

[0019] The robot 1 automatically performs a task instructed by an operator in a work space such as a factory, a warehouse, etc. The robot 1 includes a robot control device 10, an arm 11, a vision sensor 12, and a tool 13.

[0020] The robot controller 10 controls the operation of each part of the robot 1. In this embodiment, the robot controller 10 also guides the cart 2 to a specified position. The specified position is, for example, a work position where the robot 1 performs work, or a storage position for the robot 1 and the cart 2. The operator of the robot 1 specifies the specified position via an operation PC3, which will be described later.

[0021] The arm 11 is a multi-joint robot arm having a plurality of joints. The position and posture of the tip 11a of the arm 11 can be arbitrarily changed by rotating each joint.

[0022] The vision sensor 12 is attached to the tip 11a of the arm 11. The vision sensor 12 is a camera that captures images of the periphery of the robot 1. Fig. 1 shows an example in which the vision sensor 12 has a CCD camera 12A and a 3D camera 12B.

[0023] The tool 13 is detachably attached to the tip 11a of the arm 11. The tool 13 is a device for performing various tasks. Tasks performed by the robot 1 include, for example, welding, assembly such as screwing and drilling, picking such as grasping and moving parts, finishing, ultrasonic inspection, and appearance inspection, and a tool 13 appropriate for the task is attached to the tip 11a of the arm 11.

[0024] The cart 2 is, for example, an automated guided vehicle (AGV). The cart 2 includes a travel control device 20, a driving device 21, a travel assistance sensor 22, wheels 23, and a stopper 24.

[0025] The travel control device 20 controls the drive device 21 so that the carriage 2 moves (travels), stops, changes speed, etc., according to guidance from the robot 1.

[0026] The driving device 21 is a traveling motor that operates the wheels 23 of the dolly 2. In this embodiment, the wheels 23 are Mecanum wheels (registered trademark) having a plurality of rollers, and the driving device 21 has one traveling motor for each wheel 23. The driving device 21 changes the combination of the wheels 23 to be rotated and the rotation speed and rotation direction of each wheel 23 according to a control command from the travel control device 20. This allows the dolly 2 to move in any direction. Note that in other embodiments, the wheels 23 may be general wheels instead of Mecanum wheels. In this case, the driving device 21 has a traveling motor that rotates the wheels and a steering mechanism that changes the direction of the wheel axle.

[0027] The travel assistance sensor 22 is a sensor for detecting the position and traveling direction of the dolly 2 so that the dolly 2 can travel autonomously when there is no guidance from the robot 1. In this embodiment, the travel assistance sensor 22 has a position detection sensor 22A that detects the amount of movement of the dolly 2 in the horizontal direction (X-axis direction and Y-axis direction). The position detection sensor 22A is a sensor that uses the same technology as a so-called optical mouse, and detects the amount of movement of the dolly 2 in the X-axis direction and Y-axis direction by reading the pattern on the floor surface. The travel assistance sensor 22 may further have a gyrocompass 22B that detects the traveling direction of the dolly 2.

[0028] The stopper 24 is a device for fixing the cart 2 so that it does not move after the cart 2 has moved to a specified position.

[0029] The operation PC 3 is a computer operated by an operator of the robot 1. The operation PC 3 performs wireless communication with the robot 1. The operation PC 3 accepts operations from the operator and instructs the robot 1 on a designated position that is a destination and a route to the designated position (passing positions, etc.).

[0030] (Functional configuration of the robot control device) Fig. 2 is a block diagram showing the functional configuration of the robot control device and the travel control device according to the first embodiment. As shown in Fig. 2, the robot control device 10 includes a path acquisition unit 101, a sensor information acquisition unit 102, a detection unit 103, a travel instruction unit 104, and an arm control unit 105.

[0031] The route acquisition unit 101 acquires route information D1 indicating a route from the current position of the dolly 2 to a designated position from the operation PC 3.

[0032] The sensor information acquiring unit 102 acquires an image D2 captured by the vision sensor 12. For example, in this embodiment, the sensor information acquiring unit 102 acquires an image D2 captured by the CCD camera 12A of the vision sensor 12 when the dolly 2 is moving.

[0033] The detection unit 103 detects the amount of deviation of the traveling position and traveling direction (angle) of the dolly 2 from the specified route based on the mark L included in the image captured by the vision sensor 12 (CCD camera 12A). The mark L is, for example, a guide line L1 drawn on the floor surface or a two-dimensional code such as a QR code L2 (registered trademark). The mark L is attached to the traveling route of the dolly 2, the stopping position of the dolly 2 in the work area, etc. In this embodiment, an example will be described in which the mark L is attached to the floor surface, the top surface or side surface of a structure such as a support, etc., as shown in FIG. 1. In other embodiments, the mark L may be attached to other places such as a wall, a shelf, or a ceiling.

[0034] The traveling instruction unit 104 outputs a traveling instruction D4 to the carriage 2 (travel control device 20) to instruct the carriage 2 on a traveling position, stopping, speed, etc. For example, the traveling instruction unit 104 instructs the carriage 2 to correct the traveling position and traveling direction (angle) of the carriage 2 based on the amount of deviation detected by the detection unit 103.

[0035] The arm control unit 105 controls the arm 11 so that the vision sensor 12 and the tool 13 are in any position and posture. As described above, in this embodiment, the mark L is affixed to the floor surface. Therefore, while the cart 2 is moving, the arm control unit 105 controls the arm 11 so that the arm 11 is in a first posture in which the vision sensor 12 faces the floor surface (vertically downward) as shown in FIG. 1.

[0036] (Functional configuration of driving control device) As shown in FIG. 2, the driving control device 20 includes a motor control unit 201.

[0037] The motor control unit 201 controls the driving device 21 to move, stop, and change the speed of the cart 2. When there is no guidance (travel instruction) for the robot 1, the motor control unit 201 controls the driving device 21 so that the cart 2 travels to a specified position based on the sensor value (travel amount and traveling direction of the cart 2) of the travel assistance sensor 22.

[0038] If the cart 2 is moved based only on the sensor value of the travel assistance sensor 22, the travel position and traveling direction of the cart 2 may deviate from the specified path due to the influence of the condition of the floor surface of a factory or the like (unevenness, unreadable pattern, etc.). Therefore, in this embodiment, the robot control device 10 of the robot 1 detects the amount of deviation of the cart 2 from the path from the mark L, and issues a travel instruction to the travel control device 20 of the cart 2 to correct the travel position. The motor control unit 201 of the travel control device 20 controls the drive device 21 to adjust the travel position of the cart 2 according to the travel instruction from the robot control device 10.

[0039] As described above, the wheels 23 of the dolly 2 in this embodiment are Mecanum wheels. The motor control unit 201 may have a calculation unit 201A for calculating control command values ​​indicating the presence or absence of rotation, the direction of rotation, the number of rotations, etc. of the running motor of each wheel 23. For example, when the coordinates of a destination are given, the calculation unit 201A calculates control command values ​​for each running motor for moving the dolly 2 toward the destination coordinates. The motor control unit 201 controls the running motors based on the control command values ​​calculated by the calculation unit 201A.

[0040] (Robot-assisted trolley guidance) Fig. 3 is a flow chart showing an example of a process for guiding a cart by the robot according to the first embodiment. Fig. 4 is a diagram showing an example of route information according to the first embodiment. Fig. 5 is a diagram for explaining the process for guiding a robot according to the first embodiment. Here, the flow of the process for guiding a cart 2 by the robot control device 10 of the robot 1 will be explained with reference to Figs. 3 to 5.

[0041] The path acquisition unit 101 acquires path information D1 (FIG. 4) indicating the path from the current position of the cart 2 to a specified position from the operation PC 3 (step S101). The specified position is, for example, the working position of the robot 1. The specified position and path are input by the operator via the operation PC 3. As shown in FIG. 4, the path information D1 represents the coordinates (X, Y, Z) of the start position (current position), passing position, and specified position (destination), and the posture (W, P, R) of the cart 2 at each position.

[0042] When the path information D1 is acquired, the robot control device 10 starts guiding the cart 2 according to the path information D1. First, the robot 1 acquires an image D2 captured by the vision sensor 12 (step S102). As shown in Fig. 1 and Fig. 5, in this embodiment, the robot 1 captures the image D2 in a posture (first posture) in which the vision sensor 12 faces vertically downward (toward the floor surface).

[0043] Next, the detection unit 103 determines whether the mark L is detected from the acquired image D2 (step S103). If the mark L is detected (step S104; YES), the detection unit 103 detects the traveling position of the cart 2 and the amount of deviation from the route based on the detection information D3 of the mark L (step S104).

[0044] For example, as shown in FIG. 5(a), it is assumed that a guide line L1, which is a mark L, is detected from the image D2 (step S103; YES). Alternatively, as shown in FIG. 5(c), it is assumed that a QR code L2, which is a mark L, is detected from the image D2. In this case, the detection unit 103 detects a deviation amount indicating how much the detection position and angle of the mark L deviate from a reference position and a reference angle based on the detection information D3 including the coordinates (X, Y) and angle (angle R around the Z axis) of the detection position of the mark L (L1, L2) in the image D2 (step S104). The reference position is a position and angle at which the mark L should be detected when the cart 2 travels correctly along the path, and is given to the robot 1 in advance.

[0045] Also, for example, as shown in FIG. 5(c), it is assumed that a QR code L2, which is a mark L, is detected from the image D2 (step S103; YES). In this case, the detection unit 103 detects the amount of deviation of the cart 2 from the QR code L2 of the image D2, and detects the traveling position based on the reading result of the QR code L2 (step S104). For example, the QR code L2 records an area name indicating a plurality of work areas arranged in the work space. The detection unit 103 is given map information in advance that associates the area name with the position (XY coordinates) of the work area, and detects the traveling position (X, Y) of the cart 2 based on the reading result of the QR code L2 and the map information. In addition, the QR code L2 may include driving instructions such as "stop", "90° turn", and "driving speed".

[0046] Next, the travel instruction unit 104 outputs a travel instruction D4 to the carriage 2 based on the detected amount of deviation and the travel position (step S105).

[0047] For example, as shown in FIG. 5(a), when a deviation amount from the guide line L1 in the Y direction is detected, the traveling instruction unit 104 outputs a traveling instruction D4 for correcting the position of the dolly 2 in the Y direction. This traveling instruction is expressed, for example, by the relative coordinates (X, Y) of the corrected position with respect to the current position of the dolly 2. If the dolly 2 is capable of changing the traveling direction by a steering mechanism, the traveling instruction D4 may include a relative angle (R) of the corrected traveling direction with respect to the current traveling direction. Note that the traveling instruction unit 104 may output a traveling instruction D4 for correcting the traveling position of the dolly 2 when the deviation amount becomes equal to or exceeds a preset allowable value. In other words, when the deviation amount is less than the allowable value, the position of the dolly 2 does not need to be corrected.

[0048] Also, as shown in (c) of FIG. 5, when the traveling position of the dolly 2 is detected from the QR code L2, the traveling instruction unit 104 outputs a traveling instruction D4 such as coordinates indicating the next destination of the dolly 2, stopping, speed change, etc., based on the traveling position of the dolly 2 and the route information D1. For example, it is assumed that the route information D1 includes a route in which the dolly 2 moves in the X direction and then moves in the Y direction when it reaches a certain work area (area 3). In this case, when the traveling instruction unit 104 detects that the dolly 2 has reached this work area (area 3) from the reading result of the QR code L2, it outputs a traveling instruction D4 specifying the next destination coordinates to the dolly 2. Note that, when the reading result of the QR code L2 includes information indicating "stopping," "90° turn," or "traveling speed," the traveling instruction unit 104 may include an instruction to stop, turn 90°, change the traveling speed (accelerate or decelerate), etc., in the traveling instruction D4 according to the reading result. In addition, when stopping the trolley 2, the travel instruction unit 104 may detect the amount of deviation of the detected position of the QR code L2 and output a travel instruction D4 for adjusting the position and attitude of the trolley 2, and output a travel instruction D4 indicating stopping after adjusting the position and attitude of the trolley 2.

[0049] On the other hand, as shown in (b) of FIG. 5, if the detection unit 103 does not detect the landmark L from the image D2 (step S103; NO), the driving instruction unit 104 outputs a driving instruction D4 to instruct the cart 2 to drive autonomously (step S106).

[0050] Next, the robot controller 10 judges whether the cart 2 has stopped at the designated position (step S107). For example, if the robot controller 10 has instructed the cart 2 to stop at the designated position (within the work area that is the destination) in step S105 (step S107; YES), the robot controller 10 ends the guiding process of the cart 2. On the other hand, if the robot controller 10 has not instructed the cart 2 to stop at the designated position in step S105 (step S107; NO), the robot controller 10 returns to step S102 and continues the guiding process of the cart 2 (steps S102 to S107).

[0051] (Travel control processing for the trolley) 6 is a flowchart showing an example of a traveling control process of the bogie according to the first embodiment. Here, the flow of the traveling control process of the traveling control device 20 of the bogie 2 will be described with reference to FIG.

[0052] First, the motor control unit 201 acquires a traveling instruction D4 from the robot 1 (step S111).

[0053] Next, the motor control unit 201 determines whether the driving instruction D4 includes an instruction for autonomous driving (step S112).

[0054] When the driving instruction D4 includes an instruction for autonomous driving (step S112; YES), the motor control unit 201 controls the driving of the cart 2 based on the sensor value of the driving assistance sensor 22 (step S113). Since the autonomous driving control based on the sensor value is a known technique, a detailed description will be omitted.

[0055] On the other hand, if the travel instruction D4 does not include an instruction for autonomous travel (step S112; NO), the motor control unit 201 controls the travel of the dolly 2 according to the contents of the travel instruction D4 (step S114). For example, the calculation unit 201A of the motor control unit 201 calculates a control command value indicating the rotation direction and rotation speed of the travel motor (drive device 21) of each wheel 23 based on the destination coordinates, turns, travel speed, and other instructions included in the travel instruction D4. The motor control unit 201 controls the operation of each travel motor based on the calculated control command value. The travel control device 20 repeatedly executes the series of processes shown in FIG. 6 to travel the dolly 2 to a specified position.

[0056] (Action, effect) As described above, the robot 1 of this embodiment is a robot 1 mounted on a cart 2 capable of autonomous driving by a driving device 21, and is equipped with an arm 11 having a vision sensor 12 attached to its tip 11a and capable of arbitrarily changing the position and posture of the tip 11a, a detection unit 103 that detects the amount of deviation of the running position of the cart 2 from a specified route based on a landmark L included in an image D2 captured by the vision sensor 12, and a driving instruction unit 104 that instructs the cart 2 to correct the running position of the cart 2 based on the amount of deviation.

[0057] In this way, by having the robot 1 guide the cart 2, it is possible to configure the cart 2 with a minimum of equipment and cost, and to move the cart 2 along the route with high accuracy. In addition, since the vision sensor 12 already installed in the robot 1 can be used, the cost required for adding hardware on the robot 1 side can also be reduced.

[0058] Furthermore, the detection unit 103 detects the amount of deviation based on the difference between the position and angle of the mark L included in the image D2 and the reference position and reference angle.

[0059] In this way, the robot 1 can accurately detect deviations in the running position and running posture of the cart 2 based on the mark L. This allows the robot 1 to correct the running position or running posture of the cart 2 so that it is aligned with the route.

[0060] In addition, the mark L is a QR code L2 (two-dimensional code), and the detection unit 103 reads information indicating at least one of the driving position, stopping, turning, and driving speed from the QR code L2 included in the image D2, and the driving instruction unit 104 instructs the cart on at least one of the next destination, stopping, turning, and driving speed based on the information read from the QR code L2.

[0061] In this way, the robot 1 can easily detect the traveling position of the cart based on the information read from the QR code L2. Also, when information on stopping, turning, and traveling speed is recorded in the QR code L2, the robot 1 can automatically control the traveling of the cart 2 based on this information, which reduces the operator's effort of inputting detailed commands in advance.

[0062] In addition, when the image D2 does not include the landmark L, the driving instruction unit 104 of the robot 1 instructs the cart 2 to drive autonomously to a specified point on the route, and the cart 2 has a driving assistance sensor 22 capable of detecting the amount of movement and the direction of travel, and a driving control device that, when an instruction for autonomous driving is received, causes the cart 2 to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor 22.

[0063] In this way, the cart 2 can travel autonomously even in areas without the mark L. Furthermore, by restarting guidance of the robot 1 when the cart 2 reaches an area with the mark L, even if the travel position of the cart 2 deviates from the route during autonomous travel, the cart 2 can be corrected to a correct travel position along the route.

[0064] (Variation 1) In the first embodiment, an example has been described in which the robot 1 captures images in a first posture in which the vision sensor 12 faces the floor surface (vertically downward). In this modification, the robot 1 captures images in a second posture in which the vision sensor 12 faces forward in the traveling direction (horizontally), in addition to the first posture.

[0065] FIG. 7 is a diagram for explaining the guidance process of the robot according to the modified example of the first embodiment. For example, as shown in (a) of FIG. 7, when the detection unit 103 cannot detect the mark L in the first posture, the arm control unit 105 changes the position and posture of the arm 11 so as to take the second posture in which the vision sensor 12 faces forward in the traveling direction. As in the example of FIG. 1, the mark L has a QR code L2a attached to the top surface of a support and a QR code L2b attached to the side surface. When a support with such a QR code L2b attached thereto is provided ahead in the traveling direction, the detection unit 103 can detect the QR code L2b from the image D2 in the second posture. The support is installed on or near a guide line L1 attached to a floor surface.

[0066] When the detection unit 103 detects the QR code L2b in the second posture (step S103 in FIG. 3; YES), it detects the traveling position of the cart 2 and the amount of deviation from the route based on the detection information D3 of the QR code L2b (step S104 in FIG. 3). The detection information D3 includes the coordinates (X, Y, Z) of the detection position of the QR code L2b, the angle (W, P, R), and the detection size, as in the example of FIG. 7. The detection information D3 may also include the reading result of the QR code L2b. In this way, in an area where no mark L is provided on the floor surface, the robot control device 10 can switch to the second posture, detect the QR code L2b ahead in the traveling direction, and continue guiding the cart 2.

[0067] Also, as shown in FIG. 7B, when the detection size of the QR code L2b in the second posture becomes equal to or larger than a predetermined size, that is, when the vision sensor 12 approaches the support sufficiently, the arm control unit 105 switches to the first posture as shown in FIG. 7C. As described above, the support is installed on or near the mark L on the floor surface, so that the mark L on the floor surface can be detected immediately by returning to the first posture after approaching the support sufficiently. In the first posture, the distance between the vision sensor 12 and the mark L is fixed, so that the detection accuracy of the deviation amount in the detection unit 103 can be made higher than that in the second posture. In this way, by switching between the first posture and the second posture according to the detection state of the mark L, the robot 1 can maintain high detection accuracy of the deviation amount in the first posture in the area where the first mark L2a is attached on the floor surface, and can continue to guide the cart 2 by detecting the deviation amount in the second posture in the area where the first mark L2a is not on the floor surface. In other words, the period during which the cart 2 performs low-precision autonomous traveling can be reduced, making it possible to make the cart 2 travel along the route more reliably.

[0068] (Variation 2) In the first embodiment, an example was described in which the detection unit 103 detects the traveling position of the cart 2 and the like by using the guide line L1 or the QR code L2 as the mark L. In this modification, the detection unit 103 detects the traveling position of the cart 2 and the like by using a structure in a work area (an area including the path and stopping positions of the cart 2) as the mark L. The structure is, for example, a shelf, a signboard, a product, a jig, or a part of a building such as a wall or a pillar. In this modification, the robot 1 captures images in the second posture in which the vision sensor 12 is always facing forward in the traveling direction (horizontally).

[0069] In step S103 of FIG. 3, the detection unit 103 detects the shape (three-dimensional shape or two-dimensional shape) of the structure from the image D2 captured by the vision sensor 12, and determines whether the landmark L has been detected from the image D2 by comparing the shape and position of each structure, which is the landmark L, with pre-recorded reference data.

[0070] Next, in step S104 of Fig. 3, the detection unit 103 detects the traveling position of the dolly 2 based on the mark L (structure) detected from the image D2. The detection unit 103 also detects the amount of deviation of the dolly 2 from a reference position based on the detected position of the mark L (structure) in the image D2. The detection unit 103 compares the shape of the mark L detected from the image D2 with reference data (the three-dimensional shape of the mark L, or the two-dimensional shape of the mark L as seen from each direction) to detect the relative angle between the traveling direction of the dolly 2 and the mark L, and detects the amount of deviation of this relative angle from a reference angle. The reference position and the reference angle are preset positions and relative angles at which the mark L should be detected when the dolly 2 travels correctly along the route.

[0071] 3, the driving instruction unit 104 outputs driving instructions D4 to the trolley 2 based on the driving position and the amount of deviation detected by the detection unit 103. In addition, in the case where driving instructions (such as "stop", "90° turn", and "driving speed") to be performed when the landmark L is detected are recorded in the reference data, the driving instruction unit 104 may include the driving instructions read out from the reference data in the driving instructions D4 to the trolley 2.

[0072] In this way, even in an environment where the white line L1 or QR code L2 marked on the floor surface is hidden by a product or a jig, the robot 1 can detect the traveling position of the cart 2 and the amount of deviation from the path from the shape of the structure in the work area, and can guide the cart 2 to travel along the path. The robot 1 may be configured to be able to switch between the function of the first embodiment or modified example 1 and the function of modified example 2 according to the environment of the work area.

[0073] <Second embodiment> The second embodiment will be described below with reference to Fig. 8. Among the configurations of the second embodiment, configurations common to the first embodiment will be denoted by the same reference numerals and description thereof will be omitted. After moving to a work position (designated position), the robot 1 performs a task on a workpiece.

[0074] (Functional configuration of the robot control device) Fig. 8 is a block diagram showing the functional configuration of a robot control device according to the second embodiment. As shown in Fig. 8, the robot control device 10 according to the present embodiment further includes a vision correction unit 106 and a tool control unit 107.

[0075] The vision correction unit 106 corrects the user coordinate system of the workpiece based on the image captured by the vision sensor 12 to match it with the actual position and orientation of the workpiece.

[0076] The tool control unit 107 controls the tool 13 to perform a predetermined operation corresponding to the tool 13 after the arm control unit 105 moves the tip 11a (tool 13) of the arm 11 to a work position of the workpiece. The predetermined operation is, for example, welding, assembly such as screwing and drilling, picking such as gripping and moving parts, finishing, ultrasonic inspection, and appearance inspection. The tool control unit 107 has applications for controlling various tools 13. Note that these applications may be arbitrarily added or replaced when the tool 13 is replaced.

[0077] The arm control unit 105 moves the tool 13 to a working position relative to the workpiece, based on the user coordinate system corrected by the vision correction unit 106.

[0078] (Vision correction processing) Fig. 9 is a flowchart showing an example of the vision correction process of the robot according to the second embodiment. Figs. 10 to 14 are first to fourth diagrams for explaining the vision correction process of the robot according to the second embodiment. Hereinafter, the flow of the vision correction process of the robot control device 10 will be explained with reference to Figs. 9 to 14.

[0079] First, the operator arbitrarily determines the reference position UF (origin) of the user coordinate system that represents the coordinates of the work target of the robot 1 (step S201). The vision correction unit 106 sets the position specified by the operator through the operation PC3 as the reference position UF. The reference position UF is expressed by a vector (X, Y, Z, W, P, R) that indicates the position and direction from the origin P0 of the robot 1.

[0080] Next, the operator performs a teaching operation on the UF (step S202). For example, as shown in FIG. 10, the vision sensor 12 of the robot 1 captures an image of the first workpiece W1. While viewing the image, the operator specifies a position where the tool 13 will perform the operation. The vision correction unit 106 stores the specified position as a teaching point TP. In the example of FIG. 10, five positions are set as teaching points TP1 to TP5.

[0081] The operator also sets a representative position of the workpiece as seen by the robot 1 (step S203). For example, the operator specifies an arbitrary position P1 of the first workpiece W1 as the representative position. A characteristic position of the first workpiece W1, such as a position where there is a screw hole or a protrusion, is selected as the representative position. The vision correction unit 106 sets the position P1 specified by the operator as the representative position. The operator may also set multiple positions P1, P2, and P3 as in the example of FIG. 11. In this case, the vision correction unit 106 sets the average position of these positions P1, P2, and P3 as the representative position. Here, for ease of explanation, an example in which the representative position is P1 will be described.

[0082] Next, the vision correction unit 106 assumes that the reference position UF of the user coordinate system and the representative point of the workpiece as seen by the robot are known. Specifically, as shown in Fig. 11, the vision correction unit 106 obtains a vector Vn from the representative position (e.g., P1) to the reference position UF (step S204).

[0083] After the vector Vn is set, the second and subsequent workpieces (second workpiece W2) are placed and photographed by the vision sensor 12 of the robot 1. For example, as shown in FIG. 12, the position and posture of the second workpiece W2 may differ from those of the first workpiece W1. In this case, as shown in the example of FIG. 12, the teaching points TP1 to TP5 set in the first workpiece W1 are significantly shifted in the second workpiece W2. For this reason, the vision correction unit 106 of this embodiment corrects the user coordinate system according to the position and posture of the second workpiece W2 for the work after the second workpiece W2. Specifically, first, the operator, while looking at the image of the photographed second workpiece W2, specifies a representative position P1' on the second workpiece W2 corresponding to the representative position P1 of the first workpiece W1 (step S205) as shown in FIG. 13. The vision correction unit 106 sets the position specified by the operator as the representative position P1' of the second workpiece. When the operator designates a plurality of positions P1 to P3 of the first workpiece W1 in step S203, the operator also designates a plurality of positions P1' to P3' of the second workpiece W2. In this case, the vision correction unit 106 sets the average position of these positions P1' to P3' as the representative position of the second workpiece W2.

[0084] Next, the vision correction unit 106 uses the representative position P1' of the second workpiece W2 and the inverse matrix of the vector Vn to obtain a corrected reference position UF' by shifting the reference position UF in the user coordinate system (step S206). To obtain the corrected reference position UF', it is sufficient to obtain a vector Va from the origin P0 of the robot 1 to the representative position P1' and a vector Vb from the origin P0 to the corrected reference position UF' from the inverse matrix of the vector Vn. The vectors Va and Vb are vectors (X, Y, Z, W, P, R) that indicate the position and direction from the origin P0.

[0085] Further, as shown in FIG. 14, the vision correction unit 106 shifts all of the teaching points TP1-TP5 designated in step S202 by the reference position UF' at once as working points TP1'-TP5' on the second workpiece W2 (step S207). After the working points TP1'-TP5' for the second workpiece W2 are obtained, the arm control unit 105 moves the tool tip point TCP to each working point in order. Then, every time the tool tip point TCP reaches each working point, the tool control unit 107 controls the operation of the tool 13 to perform a predetermined task on the second workpiece W2. Also, for the subsequent second workpieces W2, steps S205-S207 in FIG. 9 are performed to obtain the working points TP1'-TP5' for each second workpiece W2, and then the task is performed.

[0086] (Action, effect) As described above, the robot 1 according to this embodiment further includes a vision correction unit 106 that corrects the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on the image captured by the vision sensor 12. In addition, the arm control unit 105 controls the position and posture of the arm 11 so that the tool tip point TCP attached to the tip 11a of the arm 11 moves to a working point in the corrected user coordinate system.

[0087] In this way, even if the position or posture of the second workpiece W2 is different every time a new workpiece (second workpiece W2) is placed, the robot 1 can move the tool tip point TCP to the correct working point. This can improve the accuracy of the work of the robot 1. Even if the position or posture of the carriage 2 is deviated from the designated working position, the robot 1 can accurately move the tool tip point TCP to the working point on the second workpiece W2 to perform the work. Therefore, in the guidance process of the carriage 2 in the first embodiment, an error may be allowed in the position and posture of the carriage 2 relative to the working position. Since the process of fine-tuning the position and posture of the carriage 2 at the working position can be omitted by the amount of error allowed, the time required for moving the robot 1 and the carriage 2 and adjusting the position can be shortened.

[0088] <Hardware configuration> 15 is a diagram showing an example of the hardware configuration of the robot control device and the travel control device. The computer 900 includes a CPU 901, a main storage device 902, an auxiliary storage device 903, an input / output interface 904, and a communication interface 905.

[0089] The robot control device 10 and the travel control device 20 are each implemented in a computer 900. The above-mentioned functions are stored in the auxiliary storage device 903 in the form of a program. The CPU 901 reads the program from the auxiliary storage device 903, loads it in the main storage device 902, and executes the above-mentioned processing in accordance with the program. The CPU 901 also reserves a storage area in the main storage device 902 in accordance with the program. The CPU 901 also reserves a storage area in the auxiliary storage device 903 for storing data being processed in accordance with the program.

[0090] In addition, a program for realizing all or part of the functions of the robot control device 10 and the travel control device 20 may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to perform processing by each functional unit. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. In addition, if a WWW system is used, the term "computer system" also includes a homepage providing environment (or display environment). In addition, the term "computer-readable recording medium" refers to a portable medium such as a CD, DVD, or USB, and a storage device such as a hard disk built into a computer system. In addition, if the program is distributed to the computer 900 through a communication line, the computer 900 that receives the program may deploy the program in the main storage device 902 and execute the above processing. In addition, the above program may be for realizing part of the above-mentioned functions, and may further be capable of realizing the above-mentioned functions in combination with a program already recorded in the computer system.

[0091] <Other embodiments> Although the embodiment has been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes are possible. That is, in other embodiments, the order of the above-mentioned processes may be changed as appropriate. Also, some of the processes may be executed in parallel.

[0092] <Additional Notes> The robot, the autonomous traveling robot system, the guidance method, the control method, and the program described in the above-described embodiments can be understood, for example, as follows.

[0093] (1) According to the first aspect, the robot 1 is mounted on a cart 2 capable of autonomous driving, and includes an arm 11 whose tip position and posture can be arbitrarily changed, a vision sensor 12 attached to the tip of the arm 11, a detection unit 103 that detects the amount of deviation of the cart 2 from a specified route based on a landmark L included in an image captured by the vision sensor 12, and a driving instruction unit 104 that instructs the cart 2 to correct the driving position or driving posture of the cart 2 based on the amount of deviation.

[0094] In this way, by having the robot 1 guide the cart 2, it is possible to configure the cart 2 with a minimum of equipment and cost, and to move the cart 2 along the route with high accuracy. In addition, since the vision sensor 12 already installed in the robot 1 can be used, the cost required for adding hardware on the robot 1 side can also be reduced.

[0095] (2) According to the second aspect, in the robot 1 according to the first aspect, the detection unit 103 detects the amount of deviation based on the difference between the position and angle of the mark L included in the image and the reference position and reference angle.

[0096] In this way, the robot 1 can accurately detect deviations in the running position and running posture of the cart 2 based on the mark L. This allows the robot 1 to correct the running position or running posture of the cart 2 so that it is aligned with the route.

[0097] (3) According to a third aspect, in the robot 1 relating to the first or second aspect, the mark L is a two-dimensional code L2, and the detection unit 103 reads information indicating at least one of a driving position, stopping, turning, and driving speed from the two-dimensional code L2 included in the image, and the driving instruction unit 104 instructs the cart 2 of at least one of a next destination, stopping, turning, and driving speed based on the information read from the two-dimensional code L2.

[0098] In this way, the robot 1 can easily detect the traveling position of the cart based on the information read from the two-dimensional code L2. Also, when information on stopping, turning, and traveling speed is recorded in the two-dimensional code L2, the robot 1 can automatically control the traveling of the cart 2 based on this information, which reduces the operator's effort of inputting detailed commands in advance.

[0099] (4) According to the fourth aspect, in the robot 1 relating to the first or second aspect, the landmark L is a structure within the working area of ​​the cart 2, and the detection unit 103 compares the reference data recording the shape of the structure and information indicating at least one of the driving position, stopping, turning and driving speed with the shape detected from the image D2 to detect at least one of the pieces of information, and the driving instruction unit 104 instructs the cart 2 of at least one of the next destination, stopping, turning and driving speed based on the detected information.

[0100] In this way, even in an environment where the white line L1 or QR code L2 marked on the floor surface is hidden by a product or a jig, the robot 1 can detect the traveling position of the cart 2 and whether or not it needs to stop from the shape of the structures in the work area, and appropriately guide the cart 2.

[0101] (5) According to a fifth aspect, the robot 1 relating to any one of the first to third aspects further includes an arm control unit 105 that controls the position and posture of the tip of the arm 11, and the mark L has a first mark L2a that faces vertically and a second mark L2b that faces horizontally. The arm control unit 105 controls the arm 11 to assume a first posture in which the vision sensor 12 faces vertically and can capture the first mark L2a while the cart 2 is traveling, and controls the arm 11 to assume a second posture in which the vision sensor 12 faces horizontally and can capture the second mark L2b while the cart 2 is traveling, when the first mark L2a is not included in the image captured in the first posture.

[0102] In this way, the robot 1 can switch to the second posture and detect the amount of deviation from the second mark L2b even in an area where there is no first mark L2a on the floor surface, and can continue to guide the cart 2. In other words, it is possible to reduce the period during which the cart 2 performs low-precision autonomous traveling.

[0103] (6) According to the sixth aspect, in the robot 1 relating to the fifth aspect, the detection unit 103 further detects the size of the second mark L2b when the arm 11 is in the second posture, and the arm control unit 105 switches the arm 11 from the second posture to the first posture when the size of the detected second mark L2b becomes equal to or larger than a predetermined size.

[0104] In this way, the robot 1 can maintain high detection accuracy of the first mark L2a in the first posture in an area where the first mark L2a is marked on the floor surface, which makes it possible to more reliably cause the cart 2 to travel along the route.

[0105] (7) According to a seventh aspect, the robot 1 relating to any one of the first to sixth aspects further includes a vision correction unit 106 that corrects the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12, and an arm control unit 105 that controls the position and posture of the arm 11 so that the tip point of the tool attached to the tip of the arm 11 moves to the working point in the corrected user coordinate system.

[0106] In this way, even if the position or posture of the second workpiece W2 is different every time a new workpiece (second workpiece W2) is placed, the robot 1 can move the tool tip point TCP to the correct working point. This can improve the accuracy of the work of the robot 1. Even if the position or posture of the carriage 2 is deviated from the designated working position, the robot 1 can accurately move the tool tip point TCP to the working point on the second workpiece W2 to perform the work. Therefore, in the guidance process of the carriage 2 in the first embodiment, an error may be allowed in the position and posture of the carriage 2 relative to the working position. Since the process of fine-tuning the position and posture of the carriage 2 at the working position can be omitted by the amount of error allowed, the time required for moving the robot 1 and the carriage 2 and adjusting the position can be shortened.

[0107] (8) According to an eighth aspect, the robot 1 is a robot 1 that automatically performs a specified task on a workpiece, and is equipped with an arm 11 whose tip position and posture can be arbitrarily changed, a vision sensor 12 attached to the tip of the arm 11, a tool attached to the tip of the arm 11, a vision correction unit that corrects the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12, and an arm control unit 105 that controls the position and posture of the arm 11 so that the tip point of the tool attached to the tip of the arm 11 moves to a working point in the corrected user coordinate system.

[0108] In this way, even if the position or posture of the second workpiece W2 is different every time a new workpiece (second workpiece W2) is placed, the robot 1 can move the tool tip point TCP to the correct working point. This can improve the accuracy of the work of the robot 1. Even if the position or posture of the carriage 2 is deviated from the designated working position, the robot 1 can accurately move the tool tip point TCP to the working point on the second workpiece W2 to perform the work. Therefore, in the guidance process of the carriage 2 in the first embodiment, an error may be allowed in the position and posture of the carriage 2 relative to the working position. Since the process of fine-tuning the position and posture of the carriage 2 at the working position can be omitted by the amount of error allowed, the time required for moving the robot 1 and the carriage 2 and adjusting the position can be shortened.

[0109] (9) According to a ninth aspect, an autonomous driving robot system 100 is an autonomous driving robot system 100 including a cart 2 and a robot 1 relating to any one of the first to eighth aspects, in which a driving instruction unit 104 of the robot 1 instructs the cart 2 to drive autonomously when a landmark L is not included in an image, and the cart 2 has a driving assistance sensor 22 capable of detecting an amount of movement and a traveling direction, and a driving control device 20 that, when an instruction for autonomous driving is received, causes the cart 2 to drive autonomously based on the amount of movement and the traveling direction detected by the driving assistance sensor 22.

[0110] In this way, the cart 2 can travel autonomously even in areas without the mark L. Furthermore, by restarting guidance of the robot 1 when the cart 2 reaches an area with the mark L, even if the travel position of the cart 2 deviates from the route during autonomous travel, the cart 2 can be corrected to a correct travel position along the route.

[0111] (10) According to the tenth aspect, a guidance method is a method for guiding a cart 2 capable of autonomous travel using a robot 1 having an arm 11 mounted on an autonomously travelling cart 2 and capable of arbitrarily changing the position and posture of the tip thereof, and a vision sensor 12 attached to the tip of the arm 11, the method including the steps of: detecting an amount of deviation of the cart 2 from a specified route based on a landmark L included in an image captured by the vision sensor 12; and instructing the cart 2 to correct the running position or running posture of the cart 2 based on the amount of deviation.

[0112] (11) According to the eleventh aspect, the program causes a robot 1 mounted on an autonomously movable cart 2 and having an arm 11 whose tip position and posture can be arbitrarily changed and a vision sensor 12 attached to the tip of the arm 11 to execute the steps of detecting the amount of deviation of the cart 2 from a specified path based on a landmark L contained in an image captured by the vision sensor 12 attached to the tip of the arm 11 of the robot 1, and instructing the cart 2 to correct the running position or running posture of the cart 2 based on the amount of deviation.

[0113] (12) According to a twelfth aspect, a control method is a control method for a robot 1 having an arm 11 whose tip position and posture can be arbitrarily changed, and a vision sensor 12 and a tool attached to the tip of the arm 11, and performing a specified task on a workpiece, the control method including the steps of: correcting a user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12; and controlling the position and posture of the arm 11 so that the tip point of the tool moves to a working point in the corrected user coordinate system.

[0114] (13) According to the thirteenth aspect, the program causes a robot 1 having an arm 11 whose tip position and posture can be arbitrarily changed, and a vision sensor 12 and a tool attached to the tip of the arm 11, which performs a specified task on a workpiece, to execute the following steps: correcting a user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12 attached to the tip of the arm 11 of the robot 1; and controlling the position and posture of the arm 11 so that the tip point of the tool attached to the tip of the arm 11 moves to a working point in the corrected user coordinate system. [Explanation of symbols]

[0115] 100 Autonomous Driving Robot System 1. Robot 10 Robot control device 101 Route Acquisition Department 102 Sensor information acquisition unit 103 Detection unit 104 Driving instruction unit 105 Arm control section 106 Vision correction unit 107 Tool control section 11 Arm 12 Vision Sensor 12A CCD camera 12B 3D Camera 13 Tools 2 Carts 20 Driving control device 201 Motor control unit 201A Calculation Department 21 Drive unit 22 Driving assistance sensor 22A Position Detection Sensor 22B Gyrocompass 23 wheels 24 Stopper 3 Operation PC L Mark L1 Guide line (marker) L2 QR code (two-dimensional code, landmark) L2a QR Code (First Landmark) L2b QR Code (Second Landmark)

Claims

1. A robot mounted on an autonomously movable cart, An arm whose tip position and attitude can be changed arbitrarily; A vision sensor attached to the tip of the arm; a detection unit that detects an amount of deviation of the cart with respect to a specified route based on a mark included in an image captured by the vision sensor; A travel instruction unit that instructs the carriage to correct a travel position or a travel posture of the carriage based on the deviation amount; A robot comprising:

2. the detection unit detects the amount of deviation based on a difference between a position and an angle of the mark included in the image and a reference position and a reference angle. The robot according to claim 1.

3. The mark is a two-dimensional code, The detection unit reads information indicating at least one of a driving position, a stop, a turn, and a driving speed from the two-dimensional code included in the image, The travel instruction unit instructs the carriage on at least one of a next destination, a stop, a turn, and a travel speed based on the information read from the two-dimensional code. The robot according to claim 1 or 2.

4. the landmark is a structure within a working area of ​​the carriage; The detection unit compares the shape of the structure and at least one of information indicating a driving position, a stop, a turn, and a driving speed of the structure with reference data recorded therein, and detects at least one of the information, and The travel instruction unit instructs the carriage on at least one of a next destination, a stop, a turn, and a travel speed based on the detected information. The robot according to claim 1 or 2.

5. An arm control unit that controls a position and a posture of a tip of the arm, The markings include a first marking that is oriented in a vertical direction and a second marking that is oriented in a horizontal direction; The arm control unit includes: while the carriage is traveling, the arm is controlled so that the vision sensor faces a vertical direction and assumes a first posture capable of photographing the first mark; When the carriage is traveling and the image captured in the first posture does not include a first mark, the arm is controlled so that the vision sensor faces horizontally and assumes a second posture that enables the vision sensor to capture an image of the second mark. The robot according to claim 1 or 2.

6. The detection unit further detects a size of the second mark when the arm is in the second posture, the arm control unit switches the arm from the second posture to the first posture when a size of the detected second mark becomes equal to or larger than a predetermined size. The robot according to claim 5.

7. a vision correction unit that corrects a user coordinate system of a workpiece based on an image captured by the vision sensor to match the actual position and orientation of the workpiece; an arm control unit that controls a position and a posture of the arm so that a tip point of a tool attached to a tip of the arm moves to a working point in the user coordinate system after the correction; The robot according to claim 1 or 2, further comprising:

8. A robot that automatically performs a predetermined task on a workpiece, An arm whose tip position and attitude can be changed arbitrarily; A vision sensor attached to the tip of the arm; A tool attached to the end of the arm; a vision correction unit that corrects a user coordinate system of a workpiece based on an image captured by the vision sensor to match the actual position and orientation of the workpiece; an arm control unit that controls a position and a posture of the arm so that a tip point of a tool attached to a tip of the arm moves to a working point in the user coordinate system after the correction; A robot comprising:

9. An autonomous mobile robot system comprising a cart and the robot according to claim 1 or 2, The travel instruction unit of the robot instructs the carriage to travel autonomously when the mark is not included in the image, The cart is A travel assistance sensor capable of detecting a travel distance and a travel direction; a travel control device that, when receiving the instruction for the autonomous travel, causes the cart to travel autonomously based on the amount of movement and the direction of travel detected by the travel assistance sensor; having Autonomous driving robot system.

10. A method for guiding an autonomously movable cart using a robot having an arm that is mounted on the cart and can arbitrarily change a position and a posture of a tip of the arm, and a vision sensor attached to the tip of the arm, comprising: detecting an amount of deviation of the cart from a specified route based on a mark included in an image captured by the vision sensor; instructing the carriage to correct a traveling position or a traveling attitude of the carriage based on the deviation amount; The induction method comprising:

11. A robot is mounted on an autonomous vehicle and has an arm whose tip position and attitude can be arbitrarily changed, and a vision sensor attached to the tip of the arm. detecting an amount of deviation of the cart from a specified path based on a mark included in an image captured by a vision sensor attached to the tip of the arm of the robot; instructing the carriage to correct a traveling position or a traveling attitude of the carriage based on the deviation amount; A program that executes the following.

12. A method for controlling a robot having an arm whose tip position and orientation can be arbitrarily changed, and a vision sensor and a tool attached to the tip of the arm, the robot performing a predetermined task on a workpiece, comprising the steps of: correcting a user coordinate system of a workpiece to match an actual position and orientation of the workpiece based on an image captured by the vision sensor; controlling a position and an attitude of the arm so that a tip point of the tool moves to a work point in the corrected user coordinate system; The control method includes:

13. A robot having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor and a tool attached to the tip of the arm, and performing a predetermined operation on a workpiece, correcting a user coordinate system of a workpiece to match the actual position and orientation of the workpiece based on an image captured by a vision sensor attached to the tip of the arm of the robot; controlling a position and an attitude of the arm so that a tip point of a tool attached to a tip of the arm moves to a working point in the user coordinate system after the correction; A program that executes the following.

Images