Robot, autonomous robot system, guidance method, control method, and program

Abstract

A robot according to the present invention is mounted on a vehicle which is capable of autonomous travelling, said robot comprising: an arm, orientation and the position of the tip end of which can be discretionarily changed; a vision sensor which is attached to the tip end of the arm; a detection unit which detects the amount of deviation of the vehicle from a specified path on the basis of a marker included in an image that has been captured by the vision sensor; and a travel instruction unit which, on the basis of the amount of deviation, instructs the vehicle to correct the travelling position or the travelling orientation of the vehicle.

Description

Robot, autonomous traveling robot system, guidance method, control method, and program

[0001] This disclosure relates to a robot, an autonomous traveling robot system, a guidance method, a control method, and a program. This application claims priority to Japanese Patent Application No. 2023-191569, filed on November 9, 2023, the contents of which are incorporated herein by reference.

[0002] It has been considered to mount a robot on an automated guided vehicle (AGV) and automatically transport the robot to a work position (see, for example, Patent Document 1).

[0003] Japanese Patent No. 2680298

[0004] AGVs equipped with robots navigate autonomously using guidance methods such as magnetic guidance and optical guidance. Conventional commercially available AGVs are typically equipped with sensors tailored to the guidance method and various specialized functions for autonomous driving control, making them highly sophisticated and expensive. Therefore, when a commercially available AGV is introduced to a factory, it may include functions that are not used in the factory (e.g., automatic transport scheduling and map creation functions), potentially resulting in an equipment configuration and cost that is excessive for the intended use. Furthermore, commercially available AGVs are designed to be operated in environments where the running surface is flat and stable, and in environments with uneven running surfaces, it may be difficult to navigate along the correct route.

[0005] Furthermore, if the AVG cannot move precisely to the correct working position, the relative positions of the robot and the workpiece will be misaligned, making it difficult for the robot to move the tool to the correct working point on the workpiece and perform the work, which will reduce the accuracy of the work performed by the robot.

[0006] The object of the present disclosure is to provide a robot, an autonomous mobile robot system, a guidance method, a control method, and a program that can move accurately along a route by guiding a robot mounted on a cart while configuring the cart with minimal equipment and cost.

[0007] According to one aspect of the present disclosure, the robot is mounted on an autonomously movable cart and includes an arm whose tip position and posture can be arbitrarily changed, a vision sensor attached to the tip of the arm, a detection unit that detects the amount of deviation of the cart from a specified route based on landmarks included in an image captured by the vision sensor, and a driving instruction unit that instructs the cart to correct the driving position or posture of the cart based on the amount of deviation.

[0008] According to one aspect of the present disclosure, the robot is a robot that automatically performs a predetermined task on a workpiece, and includes an arm whose tip position and posture can be arbitrarily changed, a vision sensor attached to the tip of the arm, a tool attached to the tip of the arm, a vision correction unit that corrects the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor, and an arm control unit that controls the position and posture of the arm so that the tip point of the tool attached to the tip of the arm moves to a working point in the user coordinate system after the correction.

[0009] According to one aspect of the present disclosure, an autonomous driving robot system is an autonomous driving robot system including a cart and the above-described robot, wherein the driving instruction unit of the robot instructs the cart to drive autonomously when the image does not include the landmark, and the cart has a driving assistance sensor capable of detecting the amount of movement and the direction of travel, and a driving control device that, when receiving the instruction for autonomous driving, causes the cart to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor.

[0010] According to one aspect of the present disclosure, a guidance method is a method for guiding an autonomously movable cart using a robot mounted on the cart and having an arm whose tip position and attitude can be arbitrarily changed and a vision sensor attached to the tip of the arm, the method comprising the steps of: detecting the amount of deviation of the cart from a specified route based on landmarks contained in an image captured by the vision sensor; and instructing the cart to correct the running position or attitude of the cart based on the amount of deviation.

[0011] According to one aspect of the present disclosure, the program causes a robot mounted on an autonomously movable cart, having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor attached to the tip of the arm, to perform the following steps: detecting the amount of deviation of the cart from a specified route based on landmarks contained in an image captured by the vision sensor attached to the tip of the robot's arm; and instructing the cart to correct the running position or posture of the cart based on the amount of deviation.

[0012] According to one aspect of the present disclosure, a control method is a control method for a robot having an arm whose tip position and attitude can be arbitrarily changed, and a vision sensor and tool attached to the tip of the arm, which performs a predetermined task on a workpiece, and includes the steps of correcting a user coordinate system of the workpiece to match the actual position and attitude of the workpiece based on an image captured by the vision sensor, and controlling the position and attitude of the arm so that the tip point of the tool moves to a working point in the corrected user coordinate system.

[0013] According to one aspect of the present disclosure, a program causes a robot having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor and tool attached to the tip of the arm, which performs a predetermined task on a workpiece, to execute the following steps: correcting a user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor attached to the tip of the robot's arm; and controlling the position and posture of the arm so that the tip point of the tool attached to the tip of the arm moves to a working point in the corrected user coordinate system.

[0014] According to the above aspect, by using a robot mounted on a cart to guide the cart, the cart can be configured with minimal equipment and costs, and can be moved along a route with high precision.

[0015] Furthermore, according to the above aspect, even if the robot deviates slightly from the work position, the tool can be accurately moved to the work point on the workpiece to perform the work.

[0016] 1 is a diagram illustrating an overall configuration of an autonomous mobile robot system according to a first embodiment. FIG. 2 is a block diagram illustrating the functional configuration of a robot control device and a traveling control device according to the first embodiment. FIG. 3 is a flowchart illustrating an example of a process for guiding a carriage by a robot according to the first embodiment. FIG. 4 is a diagram illustrating an example of route information according to the first embodiment. FIG. 5 is a diagram for explaining the process for guiding a robot according to the first embodiment. FIG. 6 is a flowchart illustrating an example of a process for controlling traveling of a carriage according to the first embodiment. FIG. 7 is a diagram for explaining the process for guiding a robot according to a first modified example of the first embodiment. FIG. 8 is a block diagram illustrating the functional configuration of a robot control device according to a second embodiment. FIG. 9 is a flowchart illustrating an example of a process for correcting a robot's vision according to the second embodiment. FIG. 10 is a first diagram for explaining the process for correcting a robot's vision according to the second embodiment. FIG. 11 is a second diagram for explaining the process for correcting a robot's vision according to the second embodiment. FIG. 12 is a third diagram for explaining the process for correcting a robot's vision according to the second embodiment. FIG. 13 is a fourth diagram for explaining the process for correcting a robot's vision according to the second embodiment. FIG. 14 is a fifth diagram for explaining the process for correcting a robot's vision according to the second embodiment.

[0017] First Embodiment Hereinafter, an embodiment will be described in detail with reference to FIGS.

[0018] (Overall Configuration) FIG. 1 is a diagram showing the overall configuration of an autonomous mobile robot system according to the first embodiment. The autonomous mobile robot system 100 includes a robot 1, a cart 2, and an operation PC 3.

[0019] The robot 1 automatically performs a task instructed by an operator in a work space such as a factory or a warehouse. The robot 1 includes a robot control device 10, an arm 11, a vision sensor 12, and a tool 13.

[0020] The robot control device 10 controls the operation of each part of the robot 1. In this embodiment, the robot control device 10 also guides the carriage 2 to a designated position. The designated position may be, for example, a work position where the robot 1 performs work, or a storage position for the robot 1 and carriage 2. The designated position is specified by the operator of the robot 1 via the operation PC 3, which will be described later.

[0021] The arm 11 is an articulated robot arm having a plurality of joints, and the position and orientation of the tip 11a of the arm 11 can be arbitrarily changed by rotating each joint.

[0022] The vision sensor 12 is attached to the tip 11a of the arm 11. The vision sensor 12 is a camera that captures images of the periphery of the robot 1. Fig. 1 shows an example in which the vision sensor 12 has a CCD camera 12A and a 3D camera 12B.

[0023] The tool 13 is detachably attached to the tip 11 a of the arm 11. The tool 13 is an implement for performing various tasks. Tasks performed by the robot 1 include, for example, welding, assembly tasks such as screwing and drilling, picking tasks such as grasping and moving parts, finishing, ultrasonic inspection, and visual inspection, and a tool 13 appropriate for the task is attached to the tip 11 a of the arm 11.

[0024] The carriage 2 is, for example, an automated guided vehicle (AGV) and includes a travel control device 20, a drive device 21, a travel assistance sensor 22, wheels 23, and a stopper 24.

[0025] The travel control device 20 controls the drive device 21 so that the carriage 2 moves (travels), stops, changes speed, etc. according to the guidance of the robot 1 .

[0026] The drive unit 21 is a travel motor that operates the wheels 23 of the bogie 2. In this embodiment, the wheels 23 are Mecanum wheels (registered trademark) having multiple rollers, and the drive unit 21 has one travel motor for each wheel 23. The drive unit 21 changes the combination of wheels 23 to rotate and the rotation speed and rotation direction of each wheel 23 in accordance with control commands from the travel control device 20. This allows the bogie 2 to move in any direction. Note that in other embodiments, the wheels 23 may be ordinary wheels instead of Mecanum wheels. In this case, the drive unit 21 has a travel motor that rotates the wheels and a steering mechanism that changes the direction of the wheel axles.

[0027] The travel assistance sensor 22 is a sensor for detecting the position and direction of travel of the cart 2 so that the cart 2 can travel autonomously when not being guided by the robot 1. In this embodiment, the travel assistance sensor 22 has a position detection sensor 22A that detects the amount of movement of the cart 2 in the horizontal direction (X-axis direction and Y-axis direction). The position detection sensor 22A is a sensor that uses the same technology as a so-called optical mouse, and reads the pattern on the floor surface, etc., to detect the amount of movement of the cart 2 in the X-axis direction and Y-axis direction. The travel assistance sensor 22 may also have a gyrocompass 22B that detects the direction of travel of the cart 2.

[0028] The stopper 24 is a device for fixing the carriage 2 so that it does not move after the carriage 2 has moved to a designated position.

[0029] The operation PC 3 is a computer operated by the operator of the robot 1. The operation PC 3 communicates wirelessly with the robot 1. The operation PC 3 accepts operations from the operator and instructs the robot 1 on a designated location that is a destination and a route to the designated location (such as passing positions).

[0030] (Functional Configuration of Robot Control Device) Fig. 2 is a block diagram showing the functional configuration of the robot control device and the travel control device according to embodiment 1. As shown in Fig. 2, the robot control device 10 includes a path acquisition unit 101, a sensor information acquisition unit 102, a detection unit 103, a travel instruction unit 104, and an arm control unit 105.

[0031] The route acquisition unit 101 acquires route information D1 from the operation PC 3, which indicates the route from the current position of the cart 2 to the designated position.

[0032] The sensor information acquisition unit 102 acquires the image D2 captured by the vision sensor 12. For example, in this embodiment, the sensor information acquisition unit 102 acquires the image D2 captured by the CCD camera 12A of the vision sensor 12 when the cart 2 is moving.

[0033] The detection unit 103 detects the amount of deviation of the traveling position and traveling direction (angle) of the carriage 2 from the specified route based on the markings L included in the image captured by the vision sensor 12 (CCD camera 12A). The markings L are, for example, guide lines L1 drawn on the floor or two-dimensional codes such as QR codes L2 (registered trademark). The markings L are attached to the traveling route of the carriage 2, the stopping positions of the carriage 2 within the work area, etc. Note that in this embodiment, an example will be described in which the markings L are attached to the floor, the top surface or side of a structure such as a support pillar, etc., as shown in FIG. 1 . In other embodiments, the markings L may be attached to other locations such as a wall, a shelf, or a ceiling.

[0034] The travel instruction unit 104 outputs a travel instruction D4 to the bogie 2 (travel control device 20) to instruct the bogie 2 on the travel position, stopping, speed, etc. For example, the travel instruction unit 104 instructs the bogie 2 to correct the travel position and traveling direction (angle) of the bogie 2 based on the amount of deviation detected by the detection unit 103.

[0035] The arm control unit 105 controls the arm 11 so that the vision sensor 12 and the tool 13 are in any position and posture. As described above, in this embodiment, the mark L is attached to the floor surface. Therefore, while the carriage 2 is moving, the arm control unit 105 controls the arm 11 so that the arm 11 is in a first posture in which the vision sensor 12 faces the floor surface (vertically downward) as shown in FIG. 1 .

[0036] (Functional Configuration of the Cruise Control Device) As shown in FIG. 2 , the cruise control device 20 includes a motor control unit 201 .

[0037] The motor control unit 201 controls the drive device 21 to move, stop, and change the speed of the cart 2. When there is no guidance (travel instruction) for the robot 1, the motor control unit 201 controls the drive device 21 so that the cart 2 travels to a specified position based on the sensor value (travel amount and traveling direction of the cart 2) of the travel assistance sensor 22.

[0038] If the carriage 2 is moved based solely on the sensor value of the travel assistance sensor 22, the travel position and direction of the carriage 2 may deviate from the designated path due to the influence of the condition of the floor of a factory or the like (unevenness, unreadable patterns, etc.). Therefore, in this embodiment, the robot control device 10 of the robot 1 detects the amount of deviation of the carriage 2 from the path from the mark L, and issues a travel instruction to the travel control device 20 of the carriage 2 to correct the travel position. The motor control unit 201 of the travel control device 20 controls the drive device 21 to adjust the travel position of the carriage 2 in accordance with the travel instruction from the robot control device 10.

[0039] As described above, the wheels 23 of the bogie 2 in this embodiment are Mecanum wheels. The motor control unit 201 may have a calculation unit 201A for calculating control command values ​​indicating whether or not the travel motor of each wheel 23 is rotating, the direction of rotation, the number of rotations, etc. When the calculation unit 201A is given, for example, the coordinates of a destination, it calculates control command values ​​for each travel motor for moving the bogie 2 toward the destination coordinates. The motor control unit 201 controls the travel motors based on the control command values ​​calculated by the calculation unit 201A.

[0040] (Processing for guiding a cart by a robot) Fig. 3 is a flowchart showing an example of processing for guiding a cart by a robot according to the first embodiment. Fig. 4 is a diagram showing an example of route information according to the first embodiment. Fig. 5 is a diagram for explaining processing for guiding a robot according to the first embodiment. Here, the flow of processing for guiding a cart 2 by the robot control device 10 of the robot 1 will be described with reference to Figs. 3 to 5.

[0041] The path acquisition unit 101 acquires path information D1 ( FIG. 4 ) indicating the path from the current position of the cart 2 to a designated position from the operation PC 3 (step S101). The designated position is, for example, the work position of the robot 1. The designated position and path are input by the operator via the operation PC 3. As shown in FIG. 4 , the path information D1 represents the coordinates (X, Y, Z) of the start position (current position), passing positions, and designated position (destination), respectively, and the posture (W, P, R) of the cart 2 at each position.

[0042] Upon acquiring the path information D1, the robot control device 10 starts guiding the carriage 2 according to the path information D1. First, the robot 1 acquires an image D2 captured by the vision sensor 12 (step S102). As shown in Figures 1 and 5, in this embodiment, the robot 1 captures the image D2 in a posture (first posture) in which the vision sensor 12 is facing vertically downward (toward the floor).

[0043] Next, the detection unit 103 determines whether or not a landmark L has been detected from the acquired image D2 (step S103). If the landmark L has been detected (step S104; YES), the detection unit 103 detects the traveling position of the carriage 2 and the amount of deviation from the route based on the detection information D3 of the landmark L (step S104).

[0044] For example, as shown in FIG. 5A, assume that a guide line L1, which is the mark L, is detected from image D2 (step S103; YES). Alternatively, as shown in FIG. 5C, assume that a QR code L2, which is the mark L, is detected from image D2. In this case, the detection unit 103 detects a deviation amount indicating how much the detected position and angle of the mark L deviate from a reference position and reference angle based on detection information D3 including the coordinates (X, Y) and angle (angle R around the Z axis) of the detected position of the mark L (L1, L2) in image D2 (step S104). The reference position is the position and angle at which the mark L should be detected when the cart 2 travels correctly along the path, and is given to the robot 1 in advance.

[0045] 5C, for example, assume that a QR code L2, which is a landmark L, is detected from image D2 (step S103; YES). In this case, the detection unit 103 detects the amount of deviation of the carriage 2 from the QR code L2 in image D2 and detects the traveling position based on the reading result of the QR code L2 (step S104). For example, the QR code L2 records area names indicating a plurality of work areas arranged in the work space. The detection unit 103 is provided with map information in advance that associates the area names with the positions (XY coordinates) of the work areas, and detects the traveling position (X, Y) of the carriage 2 based on the reading result of the QR code L2 and the map information. The QR code L2 may also include driving instructions such as "stop," "90-degree turn," and "traveling speed."

[0046] Next, the travel instruction unit 104 outputs a travel instruction D4 to the carriage 2 based on the detected amount of deviation and travel position (step S105).

[0047] For example, as shown in FIG. 5A, when a deviation in the Y direction from the guide line L1 is detected, the travel instruction unit 104 outputs a travel instruction D4 to correct the position of the bogie 2 in the Y direction. This travel instruction is expressed, for example, by the relative coordinates (X, Y) of the corrected position with respect to the current position of the bogie 2. If the bogie 2 is capable of changing its traveling direction using a steering mechanism, the travel instruction D4 may include a relative angle (R) of the corrected traveling direction with respect to the current traveling direction. Note that the travel instruction unit 104 may output a travel instruction D4 to correct the traveling position of the bogie 2 when the deviation is equal to or greater than a preset tolerance. In other words, if the deviation is less than the tolerance, the position of the bogie 2 does not need to be corrected.

[0048] 5C, when the travel position of the carriage 2 is detected from the QR code L2, the travel instruction unit 104 outputs a travel instruction D4, such as coordinates indicating the next destination of the carriage 2, stopping, or changing the speed, based on the travel position of the carriage 2 and the route information D1. For example, assume that the route information D1 includes a route in which the carriage 2 moves in the X direction and then moves in the Y direction upon reaching a certain work area (area 3). In this case, when the travel instruction unit 104 detects from the reading result of the QR code L2 that the carriage 2 has reached this work area (area 3), it outputs a travel instruction D4 specifying the coordinates of the next destination to the carriage 2. Note that when the reading result of the QR code L2 includes information indicating "stop," "90-degree turn," or "traveling speed," the travel instruction unit 104 may include an instruction to stop, make a 90-degree turn, change the travel speed (accelerate or decelerate), or the like in the travel instruction D4 according to the reading result. In addition, when stopping the trolley 2, the travel instruction unit 104 may detect the amount of deviation in the detected position of the QR code L2 and output a travel instruction D4 to adjust the position and attitude of the trolley 2, and after adjusting the position and attitude of the trolley 2, output a travel instruction D4 indicating stopping.

[0049] On the other hand, as shown in (b) of Figure 5, if the detection unit 103 does not detect a landmark L from the image D2 (step S103; NO), the driving instruction unit 104 outputs a driving instruction D4 to the cart 2 to drive autonomously (step S106).

[0050] Next, the robot controller 10 determines whether the carriage 2 has stopped at the designated position (step S107). For example, if the robot controller 10 has instructed the carriage 2 to stop at the designated position (within the work area that is the destination) in step S105 (step S107; YES), the robot controller 10 ends the process of guiding the carriage 2. On the other hand, if the robot controller 10 has not instructed the carriage 2 to stop at the designated position in step S105 (step S107; NO), the robot controller 10 returns to step S102 and continues the process of guiding the carriage 2 (steps S102 to S107).

[0051] (Travel Control Processing of the Bogie) Fig. 6 is a flowchart showing an example of travel control processing of the bogie according to the first embodiment. Here, the flow of the travel control processing of the travel control device 20 of the bogie 2 will be described with reference to Fig. 6 .

[0052] First, the motor control unit 201 acquires a travel instruction D4 from the robot 1 (step S111).

[0053] Next, the motor control unit 201 determines whether the driving instruction D4 includes an instruction for autonomous driving (step S112).

[0054] If the travel instruction D4 includes an instruction for autonomous travel (step S112; YES), the motor control unit 201 controls the travel of the carriage 2 based on the sensor value of the travel assistance sensor 22 (step S113). Autonomous travel control based on sensor values ​​is a known technique, and therefore a detailed description thereof will be omitted.

[0055] On the other hand, if the travel instruction D4 does not include an instruction for autonomous travel (step S112; NO), the motor control unit 201 controls the travel of the bogie 2 in accordance with the content of the travel instruction D4 (step S114). For example, the calculation unit 201A of the motor control unit 201 calculates control command values ​​indicating the rotation direction and rotation speed of the travel motors (drive devices 21) of each wheel 23 based on instructions such as destination coordinates, turns, and travel speed included in the travel instruction D4. The motor control unit 201 controls the operation of each travel motor based on the calculated control command values. The travel control device 20 repeatedly executes the series of processes shown in FIG. 6 to cause the bogie 2 to travel to a specified position.

[0056] (Action, effect) As described above, the robot 1 according to this embodiment is a robot 1 mounted on a cart 2 that can travel autonomously using a drive device 21, and is equipped with an arm 11 having a vision sensor 12 attached to its tip 11a and capable of arbitrarily changing the position and posture of the tip 11a, a detection unit 103 that detects the amount of deviation of the running position of the cart 2 from a specified route based on a mark L included in an image D2 captured by the vision sensor 12, and a travel instruction unit 104 that instructs the cart 2 to correct the running position of the cart 2 based on the amount of deviation.

[0057] In this way, by having the robot 1 guide the cart 2, it is possible to configure the cart 2 with a minimum of equipment and cost, and to move the cart 2 along the route with high precision. In addition, since the vision sensor 12 already installed in the robot 1 can be used, the cost required for adding hardware to the robot 1 can also be reduced.

[0058] Furthermore, the detection unit 103 detects the amount of deviation based on the difference between the position and angle of the mark L included in the image D2 and the reference position and reference angle.

[0059] In this way, the robot 1 can accurately detect deviations in the running position and running posture of the carriage 2 based on the mark L. This allows the robot 1 to correct the running position or running posture of the carriage 2 so that it is along the route.

[0060] In addition, the mark L is a QR code L2 (two-dimensional code), and the detection unit 103 reads information indicating at least one of the driving position, stopping, turning, and driving speed from the QR code L2 included in the image D2, and the driving instruction unit 104 instructs the cart on at least one of the next destination, stopping, turning, and driving speed based on the information read from the QR code L2.

[0061] In this way, the robot 1 can easily detect the traveling position of the carriage 2 based on the information read from the QR code L2. Furthermore, if information on stopping, turning, and traveling speed is recorded in the QR code L2, the robot 1 can automatically control the traveling of the carriage 2 based on this information, thereby reducing the operator's effort of inputting detailed commands in advance.

[0062] In addition, if the image D2 does not contain a landmark L, the driving instruction unit 104 of the robot 1 instructs the cart 2 to drive autonomously to a specified point on the route, and the cart 2 has a driving assistance sensor 22 that can detect the amount of movement and the direction of travel, and a driving control device that, when an instruction for autonomous driving is received, causes the cart 2 to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor 22.

[0063] In this way, the cart 2 can travel autonomously even in areas that are not marked with the marker L. Furthermore, by restarting guidance of the robot 1 when the cart 2 reaches an area marked with the marker L, even if the traveling position of the cart 2 deviates from the route during autonomous traveling, the cart 2 can be corrected to the correct traveling position along the route.

[0064] In the first embodiment, an example was described in which the robot 1 captures images in a first posture in which the vision sensor 12 faces the floor (vertically downward). In this modification, the robot 1 captures images in a second posture in which the vision sensor 12 faces forward in the direction of travel (horizontally), in addition to the first posture.

[0065] FIG. 7 is a diagram illustrating a robot guidance process according to a modified example of the first embodiment. For example, as shown in FIG. 7A, when the detection unit 103 is unable to detect the mark L in the first posture, the arm control unit 105 changes the position and posture of the arm 11 so that the arm 11 assumes a second posture in which the vision sensor 12 faces forward in the direction of travel. As shown in the example of FIG. 1, the mark L has a QR code L2a attached to the top surface of a support pillar and a QR code L2b attached to the side surface. When a support pillar with such a QR code L2b attached is installed ahead in the direction of travel, the detection unit 103 can detect the QR code L2b from the image D2 in the second posture. The support pillar is installed on or near a guide line L1 attached to the floor surface.

[0066] When the detection unit 103 detects the QR code L2b in the second posture (step S103 in FIG. 3 ; YES), it detects the traveling position of the carriage 2 and the amount of deviation from the path based on the detection information D3 of the QR code L2b (step S104 in FIG. 3 ). Note that the detection information D3 includes the coordinates (X, Y, Z) and angle (W, P, R) of the detected position of the QR code L2b, as in the example of FIG. 7 , and the detected size. The detection information D3 may also include the result of reading the QR code L2b. In this way, in an area where no marker L is marked on the floor, the robot control device 10 can switch to the second posture, detect the QR code L2b ahead in the traveling direction, and continue guiding the carriage 2.

[0067] Furthermore, as shown in FIG. 7B , when the detected size of the QR code L2b in the second posture becomes equal to or exceeds a predetermined size, i.e., when the vision sensor 12 approaches the support sufficiently, the arm control unit 105 switches to the first posture as shown in FIG. 7C . As described above, the support is installed on or near the mark L marked on the floor surface. Therefore, by returning to the first posture after approaching the support sufficiently, the mark L marked on the floor surface can be immediately detected. In the first posture, the distance between the vision sensor 12 and the mark L is fixed, so the detection accuracy of the deviation amount by the detection unit 103 can be higher than in the second posture. In this way, by switching between the first posture and the second posture depending on the detection state of the mark L, the robot 1 can maintain high deviation detection accuracy in the first posture in areas where the first mark L2a is marked on the floor surface, and can detect deviation in the second posture in areas where the first mark L2a is not on the floor surface, thereby continuously guiding the carriage 2. In other words, the period during which the carriage 2 performs low-precision autonomous traveling can be reduced, making it possible to more reliably cause the carriage 2 to travel along the route.

[0068] (Variation 2) In the first embodiment, an example was described in which the detection unit 103 detects the traveling position of the cart 2 and the like by using a guide line L1 or a QR code L2 as the mark L. In this variation, the detection unit 103 detects the traveling position of the cart 2 and the like by using a structure within the work area (an area including the path and stopping positions of the cart 2) as the mark L. Examples of the structure include a shelf, a signboard, a product, a jig, or part of a building such as a wall or a pillar. In this variation, the robot 1 always captures images in a second posture in which the vision sensor 12 is always facing forward in the traveling direction (horizontally).

[0069] In step S103 of Figure 3, the detection unit 103 detects the shape (three-dimensional shape or two-dimensional shape) of the structure from the image D2 captured by the vision sensor 12, and determines whether the landmark L has been detected from the image D2 by comparing the shape and position of each structure that is the landmark L with pre-recorded reference data.

[0070] Next, in step S104 of Fig. 3, the detection unit 103 detects the traveling position of the carriage 2 based on the mark L (structure) detected from the image D2. The detection unit 103 also detects the amount of deviation of the carriage 2 from a reference position based on the detected position of the mark L (structure) in the image D2. The detection unit 103 compares the shape of the mark L detected from the image D2 with reference data (the three-dimensional shape of the mark L, or the two-dimensional shape of the mark L as seen from each direction), detects the relative angle between the traveling direction of the carriage 2 and the mark L, and detects the amount of deviation of this relative angle from a reference angle. The reference position and reference angle are preset positions and relative angles at which the mark L should be detected when the carriage 2 is traveling correctly along the route.

[0071] 3, the driving instruction unit 104 outputs driving instructions D4 to the carriage 2 based on the driving position and the amount of deviation detected by the detection unit 103. Furthermore, if the reference data records driving instructions (such as "stop," "90° turn," and "driving speed") to be implemented when the landmark L is detected, the driving instruction unit 104 may include the driving instructions read out from the reference data in the driving instructions D4 to the carriage 2.

[0072] In this way, even in an environment where the white line L1 or QR code L2 marked on the floor surface is hidden by a product, a jig, or the like, the robot 1 can detect the traveling position of the cart 2 and the amount of deviation from the path from the shape of the structure in the work area, and can guide the cart 2 to travel along the path. The robot 1 may be configured to be able to switch between the functions of the first embodiment or modified example 1 and the functions of modified example 2 according to the environment of the work area.

[0073] Second Embodiment A second embodiment will be described below with reference to Fig. 8. Among the components of the second embodiment, components common to the first embodiment are denoted by the same reference numerals, and descriptions thereof will be omitted. After moving to a work position (designated position), the robot 1 performs work on a workpiece.

[0074] (Functional Configuration of Robot Control Device) Fig. 8 is a block diagram showing the functional configuration of a robot control device according to embodiment 2. As shown in Fig. 8, the robot control device 10 according to this embodiment further includes a vision correction unit 106 and a tool control unit 107.

[0075] The vision correction unit 106 corrects the user coordinate system of the workpiece based on the image captured by the vision sensor 12 to match the actual position and orientation of the workpiece.

[0076] After the arm control unit 105 moves the tip 11a of the arm 11 (tool 13) to a work position for the workpiece, the tool control unit 107 controls the tool 13 to perform a predetermined task corresponding to the tool 13. The predetermined tasks include, for example, welding, assembly tasks such as screwing and drilling, picking tasks such as grasping and moving parts, finishing, ultrasonic inspection, and visual inspection. The tool control unit 107 has applications for controlling various tools 13. Note that these applications may be optionally added or replaced when the tool 13 is replaced.

[0077] The arm control unit 105 moves the tool 13 to a working position relative to the workpiece based on the user coordinate system corrected by the vision correction unit 106.

[0078] (Vision Correction Processing) Fig. 9 is a flowchart showing an example of the vision correction processing of the robot according to the second embodiment. Figs. 10 to 14 are first to fourth diagrams for explaining the vision correction processing of the robot according to the second embodiment. The flow of the vision correction processing of the robot control device 10 will be described below with reference to Figs. 9 to 14.

[0079] First, the operator arbitrarily determines the reference position UF (origin) of the user coordinate system that represents the coordinates of the work target of the robot 1 (step S201). The vision correction unit 106 sets the position specified by the operator via the operation PC 3 as the reference position UF. The reference position UF is expressed by vectors (X, Y, Z, W, P, R) that indicate the position and direction of the robot 1 from the origin P0.

[0080] Next, the operator performs a teaching operation on the UF (step S202). For example, as shown in FIG. 10, the vision sensor 12 of the robot 1 captures an image of the first workpiece W1. While viewing the image, the operator specifies the position where the tool 13 will perform the operation. The vision correction unit 106 stores the specified position as a teaching point TP. In the example of FIG. 10, five positions are set as teaching points TP1 to TP5.

[0081] The operator also sets a representative position of the workpiece as seen from the robot 1 (step S203). For example, the operator designates an arbitrary position P1 of the first workpiece W1 as the representative position. A characteristic position of the first workpiece W1, such as a position where a screw hole or a protrusion is located, is selected as the representative position. The vision correction unit 106 sets the position P1 designated by the operator as the representative position. The operator may also set multiple positions P1, P2, and P3, as in the example of FIG. 11. In this case, the vision correction unit 106 sets the average position of these positions P1, P2, and P3 as the representative position. For simplicity of explanation, an example in which the representative position is P1 will be described here.

[0082] Next, the vision correction unit 106 sets the reference position UF in the user coordinate system and the representative point of the workpiece as seen from the robot as known. Specifically, as shown in Figure 11, the vision correction unit 106 calculates a vector Vn from the representative position (for example, P1) to the reference position UF (step S204).

[0083] After setting the vector Vn, the second and subsequent workpieces (second workpiece W2) are placed and photographed by the vision sensor 12 of the robot 1. For example, as shown in FIG. 12, the position and orientation of the second workpiece W2 may differ from those of the first workpiece W1. In this case, as shown in the example of FIG. 12, the teaching points TP1 to TP5 set for the first workpiece W1 may be significantly offset on the second workpiece W2. Therefore, in this embodiment, the vision correction unit 106 corrects the user coordinate system for tasks after the second workpiece W2 to match the position and orientation of the second workpiece W2. Specifically, as shown in FIG. 13, the operator first specifies a representative position P1' on the second workpiece W2 corresponding to the representative position P1 of the first workpiece W1 while viewing the photographed image of the second workpiece W2 (step S205). The vision correction unit 106 then sets the position specified by the operator as the representative position P1' of the second workpiece W2. When the operator designates multiple positions P1 to P3 for the first workpiece W1 in step S203, the operator also designates multiple positions P1' to P3' for the second workpiece W2. In this case, the vision correction unit 106 sets the average position of these positions P1' to P3' as the representative position of the second workpiece W2.

[0084] Next, the vision correction unit 106 calculates a corrected reference position UF' by shifting the reference position UF in the user coordinate system using the representative position P1' of the second workpiece W2 and the inverse matrix of the vector Vn (step S206). To calculate the corrected reference position UF', it is sufficient to calculate a vector Va from the origin P0 of the robot 1 to the representative position P1' and a vector Vb from the origin P0 to the corrected reference position UF' from the inverse matrix of the vector Vn. The vectors Va and Vb are vectors (X, Y, Z, W, P, R) that indicate the position and direction from the origin P0.

[0085] Furthermore, as shown in FIG. 14, the vision correction unit 106 simultaneously shifts all of the teaching points TP1 to TP5 specified in step S202 using the reference position UF' to set them as working points TP1' to TP5' on the second workpiece W2 (step S207). After the working points TP1' to TP5' for the second workpiece W2 are obtained, the arm control unit 105 moves the tool tip point TCP to each working point in turn. Then, each time the tool tip point TCP reaches each working point, the tool control unit 107 controls the operation of the tool 13 to perform a predetermined task on the second workpiece W2. Furthermore, for subsequent second workpieces W2, steps S205 to S207 of FIG. 9 are performed to obtain working points TP1' to TP5' for each second workpiece W2 before performing the task.

[0086] (Functions and Effects) As described above, the robot 1 according to this embodiment further includes a vision correction unit 106 that corrects the user coordinate system of the workpiece to match the actual position and orientation of the workpiece based on the image captured by the vision sensor 12. In addition, the arm control unit 105 controls the position and orientation of the arm 11 so that the tool tip point TCP attached to the tip 11a of the arm 11 moves to a working point in the corrected user coordinate system.

[0087] By doing so, the robot 1 can move the tool tip point TCP to the correct working point each time a new workpiece (second workpiece W2) is placed, even if the position or orientation of the second workpiece W2 is different. This improves the accuracy of the work performed by the robot 1. Furthermore, even if the position or orientation of the carriage 2 deviates from the designated working position, the robot 1 can accurately move the tool tip point TCP to the working point on the second workpiece W2 to perform the work. Therefore, in the guidance process for the carriage 2 in the first embodiment, an error may be allowed in the position and orientation of the carriage 2 relative to the working position. By allowing the error, the process of fine-tuning the position and orientation of the carriage 2 at the working position can be omitted, thereby shortening the time required to move and adjust the position of the robot 1 and the carriage 2.

[0088] 15 is a diagram showing an example of the hardware configuration of the robot control device and the travel control device. The computer 900 includes a CPU 901, a main memory device 902, an auxiliary memory device 903, an input / output interface 904, and a communication interface 905.

[0089] The robot controller 10 and the travel controller 20 are each implemented in a computer 900. The above-described functions are stored in the auxiliary storage device 903 in the form of a program. The CPU 901 reads the program from the auxiliary storage device 903, loads it into the main storage device 902, and executes the above-described processing in accordance with the program. The CPU 901 also allocates a storage area in the main storage device 902 in accordance with the program. The CPU 901 also allocates a storage area in the auxiliary storage device 903 for storing data being processed in accordance with the program.

[0090] Alternatively, a program for implementing all or part of the functions of the robot controller 10 and the driving controller 20 may be recorded on a computer-readable recording medium, and the program may be loaded into a computer system and executed to perform processing by each functional unit. The term "computer system" as used herein includes hardware such as an OS and peripheral devices. Furthermore, if a WWW system is used, the term "computer system" also includes a website provision environment (or display environment). Furthermore, the term "computer-readable recording medium" refers to portable media such as CDs, DVDs, and USBs, as well as storage devices such as hard disks built into the computer system. Furthermore, if the program is distributed to the computer 900 via a communication line, the computer 900 may load the program into the main storage device 902 and execute the processing described above. Furthermore, the program may be for implementing part of the functions described above, or may be capable of implementing the functions described above in combination with a program already stored in the computer system.

[0091] Other Embodiments Although the embodiments have been described in detail above with reference to the drawings, the specific configuration is not limited to the above, and various design changes are possible. That is, in other embodiments, the order of the above-described processes may be changed as appropriate. Furthermore, some processes may be executed in parallel.

[0092] <Additional Notes> The robot, autonomous traveling robot system, guidance method, control method, and program described in the above-described embodiments can be understood, for example, as follows.

[0093] (1) According to the first aspect, the robot 1 is mounted on an autonomously movable cart 2, and includes an arm 11 whose tip position and posture can be arbitrarily changed, a vision sensor 12 attached to the tip of the arm 11, a detection unit 103 that detects the amount of deviation of the cart 2 from a specified route based on a mark L included in an image captured by the vision sensor 12, and a driving instruction unit 104 that instructs the cart 2 to correct the driving position or driving posture of the cart 2 based on the amount of deviation.

[0094] In this way, by having the robot 1 guide the cart 2, it is possible to configure the cart 2 with a minimum of equipment and cost, and to move the cart 2 along the route with high precision. In addition, since the vision sensor 12 already installed in the robot 1 can be used, the cost required for adding hardware to the robot 1 can also be reduced.

[0095] (2) According to the second aspect, in the robot 1 according to the first aspect, the detection unit 103 detects the amount of deviation based on the difference between the position and angle of the mark L included in the image and the reference position and reference angle.

[0096] In this way, the robot 1 can accurately detect deviations in the running position and running posture of the carriage 2 based on the mark L. This allows the robot 1 to correct the running position or running posture of the carriage 2 so that it is along the route.

[0097] (3) According to a third aspect, in the robot 1 relating to the first or second aspect, the mark L is a two-dimensional code L2, and the detection unit 103 reads information indicating at least one of the running position, stopping, turning, and running speed from the two-dimensional code L2 included in the image, and the running instruction unit 104 instructs the cart 2 on at least one of the next destination, stopping, turning, and running speed based on the information read from the two-dimensional code L2.

[0098] In this way, the robot 1 can easily detect the traveling position of the carriage 2 based on the information read from the two-dimensional code L2. Furthermore, if information on stopping, turning, and traveling speed is recorded in the two-dimensional code L2, the robot 1 can automatically control the traveling of the carriage 2 based on this information, thereby reducing the operator's effort of inputting detailed commands in advance.

[0099] (4) According to the fourth aspect, in the robot 1 relating to the first or second aspect, the landmark L is a structure within the working area of ​​the cart 2, and the detection unit 103 compares the shape of the structure and reference data recording information indicating at least one of the traveling position, stopping, turning, and traveling speed with the shape detected from the image D2 to detect at least one of the pieces of information, and the traveling instruction unit 104 instructs the cart 2 on at least one of the next destination, stopping, turning, and traveling speed based on the detected information.

[0100] In this way, even in an environment where the white lines L1 and QR codes L2 on the floor are hidden by products or jigs, the robot 1 can detect the running position of the cart 2 and whether it needs to stop from the shape of the structures in the work area, and can appropriately guide the cart 2.

[0101] (5) According to the fifth aspect, the robot 1 according to any one of the first to third aspects further includes an arm control unit 105 that controls the position and posture of the tip of the arm 11, wherein the mark L has a first mark L2a that faces vertically and a second mark L2b that faces horizontally, and the arm control unit 105 controls the arm 11 to assume a first posture in which the vision sensor 12 faces vertically and can photograph the first mark L2a while the cart 2 is traveling, and controls the arm 11 to assume a second posture in which the vision sensor 12 faces horizontally and can photograph the second mark L2b when the cart 2 is traveling and the image photographed in the first posture does not include the first mark L2a.

[0102] In this way, the robot 1 can switch to the second posture and detect the amount of deviation from the second mark L2b even in an area where there is no first mark L2a on the floor surface, and can therefore continue to guide the cart 2. In other words, it is possible to reduce the period during which the cart 2 performs autonomous traveling with low accuracy.

[0103] (6) According to the sixth aspect, in the robot 1 relating to the fifth aspect, the detection unit 103 further detects the size of the second mark L2b when the arm 11 is in the second posture, and the arm control unit 105 switches the arm 11 from the second posture to the first posture when the size of the detected second mark L2b becomes equal to or larger than a predetermined size.

[0104] In this way, the robot 1 can maintain high detection accuracy of the first mark L2a in the first posture in an area where the first mark L2a is marked on the floor surface, thereby enabling the cart 2 to travel along the route more reliably.

[0105] (7) According to the seventh aspect, the robot 1 relating to any one of the first to sixth aspects further includes a vision correction unit 106 that corrects the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12, and an arm control unit 105 that controls the position and posture of the arm 11 so that the tip point of the tool attached to the tip of the arm 11 moves to the working point in the corrected user coordinate system.

[0106] By doing so, the robot 1 can move the tool tip point TCP to the correct working point each time a new workpiece (second workpiece W2) is placed, even if the position or orientation of the second workpiece W2 is different. This improves the accuracy of the work performed by the robot 1. Furthermore, even if the position or orientation of the carriage 2 deviates from the designated working position, the robot 1 can accurately move the tool tip point TCP to the working point on the second workpiece W2 to perform the work. Therefore, in the guidance process for the carriage 2 in the first embodiment, an error may be allowed in the position and orientation of the carriage 2 relative to the working position. By allowing the error, the process of fine-tuning the position and orientation of the carriage 2 at the working position can be omitted, thereby shortening the time required to move and adjust the position of the robot 1 and the carriage 2.

[0107] (8) According to the eighth aspect, the robot 1 is a robot 1 that automatically performs a predetermined task on a workpiece, and includes an arm 11 whose tip position and posture can be arbitrarily changed, a vision sensor 12 attached to the tip of the arm 11, a tool attached to the tip of the arm 11, a vision correction unit that corrects the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12, and an arm control unit 105 that controls the position and posture of the arm 11 so that the tip point of the tool attached to the tip of the arm 11 moves to the working point in the corrected user coordinate system.

[0108] By doing so, the robot 1 can move the tool tip point TCP to the correct working point each time a new workpiece (second workpiece W2) is placed, even if the position or orientation of the second workpiece W2 is different. This improves the accuracy of the work performed by the robot 1. Furthermore, even if the position or orientation of the carriage 2 deviates from the designated working position, the robot 1 can accurately move the tool tip point TCP to the working point on the second workpiece W2 to perform the work. Therefore, in the guidance process for the carriage 2 in the first embodiment, an error may be allowed in the position and orientation of the carriage 2 relative to the working position. By allowing the error, the process of fine-tuning the position and orientation of the carriage 2 at the working position can be omitted, thereby shortening the time required to move and adjust the position of the robot 1 and the carriage 2.

[0109] (9) According to a ninth aspect, the autonomous driving robot system 100 is an autonomous driving robot system 100 comprising a cart 2 and a robot 1 relating to any one of the first to eighth aspects, wherein the driving instruction unit 104 of the robot 1 instructs the cart 2 to drive autonomously when the image does not include a landmark L, and the cart 2 has a driving assistance sensor 22 capable of detecting the amount of movement and the direction of travel, and a driving control device 20 that, when receiving an instruction for autonomous driving, causes the cart 2 to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor 22.

[0110] In this way, the cart 2 can travel autonomously even in areas that are not marked with the marker L. Furthermore, by restarting guidance of the robot 1 when the cart 2 reaches an area marked with the marker L, even if the traveling position of the cart 2 deviates from the route during autonomous traveling, the cart 2 can be corrected to the correct traveling position along the route.

[0111] (10) According to the tenth aspect, the guidance method is a method for guiding a cart 2 using a robot 1 mounted on an autonomously movable cart 2 and having an arm 11 whose tip position and posture can be arbitrarily changed and a vision sensor 12 attached to the tip of the arm 11, and includes the steps of detecting the amount of deviation of the cart 2 from a specified route based on a landmark L included in an image captured by the vision sensor 12, and instructing the cart 2 to correct the running position or running posture of the cart 2 based on the amount of deviation.

[0112] (11) According to the eleventh aspect, the program causes a robot 1, which is mounted on an autonomously movable cart 2 and has an arm 11 whose tip position and posture can be arbitrarily changed and a vision sensor 12 attached to the tip of the arm 11, to execute the steps of detecting the amount of deviation of the cart 2 from a specified route based on a landmark L included in an image captured by the vision sensor 12 attached to the tip of the arm 11 of the robot 1, and instructing the cart 2 to correct the running position or running posture of the cart 2 based on the amount of deviation.

[0113] (12) According to the twelfth aspect, a control method is a control method for a robot 1 having an arm 11 whose tip position and posture can be arbitrarily changed, a vision sensor 12 and a tool attached to the tip of the arm 11, and performing a predetermined task on a workpiece, the control method including the steps of: correcting the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12; and controlling the position and posture of the arm 11 so that the tip point of the tool moves to the task point in the corrected user coordinate system.

[0114] (13) According to the thirteenth aspect, the program causes a robot 1 having an arm 11 whose tip position and posture can be arbitrarily changed, a vision sensor 12 attached to the tip of the arm 11, and a tool, which performs a predetermined task on a workpiece, to execute the following steps: correcting the user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor 12 attached to the tip of the arm 11 of the robot 1; and controlling the position and posture of the arm 11 so that the tip point of the tool attached to the tip of the arm 11 moves to the working point in the corrected user coordinate system.

[0115] According to the above aspect, by using a robot mounted on a cart to guide the cart, the cart can be configured with minimal equipment and costs, and can be moved along a route with high precision.

[0116] Furthermore, according to the above aspect, even if the robot deviates slightly from the work position, the tool can be accurately moved to the work point on the workpiece to perform the work.

[0117] 100 Autonomous traveling robot system 1 Robot 10 Robot control device 101 Route acquisition unit 102 Sensor information acquisition unit 103 Detection unit 104 Travel instruction unit 105 Arm control unit 106 Vision correction unit 107 Tool control unit 11 Arm 12 Vision sensor 12A CCD camera 12B 3D camera 13 Tool 2 Cart 20 Travel control device 201 Motor control unit 201A Calculation unit 21 Drive device 22 Travel assistance sensor 22A Position detection sensor 22B Gyrocompass 23 Wheel 24 Stopper 3 Operation PC L Mark L1 Guide line (mark) L2 QR code (two-dimensional code, mark) L2a QR code (first mark) L2b QR code (second mark)

Claims

1. A robot mounted on an autonomously movable cart, comprising: an arm whose tip position and attitude can be arbitrarily changed; a vision sensor attached to the tip of the arm; a detection unit that detects the amount of deviation of the cart from a specified route based on landmarks contained in images captured by the vision sensor; and a driving instruction unit that instructs the cart to correct the driving position or attitude of the cart based on the amount of deviation.

2. The robot according to claim 1, wherein the detection unit detects the amount of deviation based on a difference between the position and angle of the mark included in the image and a reference position and reference angle.

3. The robot described in claim 1 or 2, wherein the landmark is a two-dimensional code, the detection unit reads information indicating at least one of a driving position, stopping, turning, and driving speed from the two-dimensional code included in the image, and the driving instruction unit instructs the cart as to at least one of a next destination, stopping, turning, and driving speed based on the information read from the two-dimensional code.

4. The robot described in claim 1 or 2, wherein the landmark is a structure within the working area of ​​the cart, the detection unit detects at least one of the information by comparing the shape of the structure and reference data recorded therein indicating at least one of a traveling position, stopping, turning, and traveling speed with the shape detected from the image, and the driving instruction unit instructs the cart as to at least one of a next destination, stopping, turning, and traveling speed based on the detected information.

5. The robot described in claim 1 or 2, further comprising an arm control unit that controls the position and posture of the tip of the arm, wherein the mark has a first mark facing vertically and a second mark facing horizontally, and the arm control unit controls the arm to assume a first posture in which the vision sensor faces vertically and can photograph the first mark while the cart is traveling, and controls the arm to assume a second posture in which the vision sensor faces horizontally and can photograph the second mark while the cart is traveling, when the first mark is not included in the image captured in the first posture.

6. The robot described in claim 5, wherein the detection unit further detects a size of the second mark when the arm is in the second posture, and the arm control unit switches the arm from the second posture to the first posture when the detected size of the second mark becomes equal to or larger than a predetermined size.

7. The robot described in claim 1 or 2, further comprising: a vision correction unit that corrects the user coordinate system of the workpiece to match the actual position and orientation of the workpiece based on the image captured by the vision sensor; and an arm control unit that controls the position and orientation of the arm so that the tip point of a tool attached to the end of the arm moves to a working point in the corrected user coordinate system.

8. A robot that automatically performs a specified task on a workpiece, comprising: an arm whose tip position and posture can be arbitrarily changed; a vision sensor attached to the tip of the arm; a tool attached to the tip of the arm; a vision correction unit that corrects a user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor; and an arm control unit that controls the position and posture of the arm so that the tip point of the tool attached to the tip of the arm moves to a working point in the corrected user coordinate system.

9. An autonomous driving robot system comprising a cart and the robot according to claim 1 or 2, wherein the driving instruction unit of the robot instructs the cart to drive autonomously when the image does not include the landmark, and the cart has: a driving assistance sensor capable of detecting an amount of movement and a direction of travel; and a driving control device that, when the instruction for autonomous driving is received, causes the cart to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor.

10. A method for guiding an autonomously movable cart using a robot having an arm mounted on the cart and capable of arbitrarily changing the position and posture of the tip thereof, and a vision sensor attached to the tip of the arm, the method comprising the steps of: detecting an amount of deviation of the cart from a specified route based on landmarks contained in an image captured by the vision sensor; and instructing the cart to correct the running position or posture of the cart based on the amount of deviation.

11. A program that causes a robot mounted on an autonomously movable cart, having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor attached to the tip of the arm, to execute the following steps: detecting the amount of deviation of the cart from a specified route based on landmarks contained in an image captured by the vision sensor attached to the tip of the robot's arm; and instructing the cart to correct its traveling position or posture based on the amount of deviation.

12. A method for controlling a robot having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor and tool attached to the tip of the arm, which performs a specified task on a workpiece, the method comprising the steps of: correcting a user coordinate system of the workpiece to match the actual position and posture of the workpiece based on an image captured by the vision sensor; and controlling the position and posture of the arm so that the tip point of the tool moves to a working point in the corrected user coordinate system.

13. A program that causes a robot having an arm whose tip position and posture can be arbitrarily changed, and a vision sensor and tool attached to the tip of the arm, and which performs a specified task on a workpiece, to execute the following steps: correcting a user coordinate system of a workpiece to match the actual position and posture of the workpiece based on an image captured by a vision sensor attached to the tip of the robot's arm; and controlling the position and posture of the arm so that the tip point of the tool attached to the tip of the arm moves to a working point in the corrected user coordinate system.

Images