Robot, autonomous travel robot system, guidance method, control method, and program

Abstract

A robot mounted on a trolley capable of autonomously traveling, the robot including: an arm capable of arbitrarily changing the position and orientation of the tip; the visual sensor is arranged at the front end of the arm; a detection unit that detects an amount of deviation of the carriage with respect to a specified path on the basis of a mark included in an image captured by the vision sensor; and a travel instruction unit that instructs the trolley to correct the travel position or the travel posture of the trolley on the basis of the deviation amount.

Description

Technical Field

[0001] This disclosure relates to robots, autonomous robot systems, guidance methods, control methods, and procedures. This application claims priority based on Japanese Patent Application No. 2023-191569, filed on November 9, 2023, the contents of which are incorporated herein by reference. Background Technology

[0002] People are considering mounting robots on automated guided vehicles (AGVs) to automatically transport them to the work location (see, for example, Patent Document 1). Existing technical documents Patent documents

[0003] Patent Document 1: Japanese Patent No. 2680298. Summary of the Invention The problem that the invention aims to solve

[0004] AGVs equipped with robots travel autonomously using guidance methods such as magnetic guidance and optical guidance. Generally, commercially available AGVs are specifically equipped with sensors corresponding to the guidance method and various functions for autonomous driving control, making them highly functional and expensive devices. Therefore, when introducing commercially available AGVs into a factory, they may include functions that the factory does not use (such as automated material handling scheduling and map generation functions), and the equipment configuration and cost may be excessive relative to the intended use. In addition, commercially available AGVs are designed for use in environments with flat and stable terrain; in environments with uneven terrain, they may struggle to travel on the correct path.

[0005] Furthermore, if the AGV cannot move precisely to the correct working position, the relative position between the robot and the workpiece will deviate, making it difficult for the robot to move the tool to the correct working point on the workpiece for operation. As a result, the accuracy of the robot's operation will decrease.

[0006] The purpose of this disclosure is to provide a robot, an autonomous robot system, a guidance method, a control method, and a program that enable a robot mounted on a trolley to move with high precision along a path while constructing the trolley with minimal equipment and cost. Methods for solving problems

[0007] According to one aspect of this disclosure, a robot is mounted on a trolley capable of autonomous movement. The robot comprises: an arm capable of arbitrarily changing the position and posture of its front end; a vision sensor mounted on the front end of the arm; a detection unit that detects the amount of deviation of the trolley relative to a specified path based on markers contained in an image captured by the vision sensor; and a driving instruction unit that instructs the trolley to correct its driving position or driving posture based on the amount of deviation.

[0008] According to one aspect of this disclosure, a robot automatically performs a predetermined task on a workpiece. The robot comprises: an arm capable of arbitrarily changing the position and posture of its front end; a vision sensor mounted on the front end of the arm; a tool mounted on the front end of the arm; a vision correction unit that corrects the user coordinate system of the workpiece based on images captured by the vision sensor to match the actual position and posture of the workpiece; and an arm control unit that controls the position and posture of the arm so that the tip of the tool mounted on the front end of the arm moves to the work point in the corrected user coordinate system.

[0009] According to one aspect of this disclosure, an autonomous driving robot system includes a trolley and the robot described above. In the autonomous driving robot system, when the driving instruction part of the robot does not contain the marker in the image, it instructs the trolley to drive autonomously. The trolley has: a driving assistance sensor capable of detecting the amount of movement and the direction of travel; and a driving control device that, upon receiving the autonomous driving instruction, enables the trolley to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor.

[0010] According to one aspect of this disclosure, a guidance method is a method for guiding a trolley that uses a robot. The robot is mounted on the autonomously moving trolley and has an arm capable of arbitrarily changing the position and posture of its front end, and a vision sensor mounted on the front end of the arm. The guidance method includes the following steps: detecting a deviation of the trolley relative to a specified path based on markers contained in an image captured by the vision sensor; and instructing the trolley to correct its driving position or driving posture based on the deviation.

[0011] According to one aspect of this disclosure, a procedure is provided to cause a robot mounted on an autonomously moving trolley, having an arm capable of arbitrarily changing the position and posture of its front end, and a vision sensor mounted on the front end of the arm, to perform the following steps: detecting a deviation of the trolley relative to a specified path based on markers contained in an image captured by the vision sensor mounted on the front end of the robot's arm; and instructing the trolley to correct its driving position or driving posture based on the deviation.

[0012] According to one aspect of this disclosure, a control method is provided for a robot, the robot having an arm capable of arbitrarily changing the position and posture of its front end, and a vision sensor and tool mounted on the front end of the arm, and performing a predetermined operation on a workpiece. The control method includes the following steps: calibrating the user coordinate system of the workpiece based on images captured by the vision sensor to match the actual position and posture of the workpiece; and controlling the position and posture of the arm so that the tip of the tool moves to the work point in the calibrated user coordinate system.

[0013] According to one aspect of this disclosure, a procedure is provided to enable a robot having an arm capable of arbitrarily changing the position and posture of its front end, and a vision sensor and tool mounted on the front end of the arm, and performing a predetermined operation on a workpiece, to perform the following steps: calibrating the user coordinate system of the workpiece based on an image captured by the vision sensor mounted on the front end of the robot's arm to match the actual position and posture of the workpiece; and controlling the position and posture of the arm such that the tip of the tool mounted on the front end of the arm moves to the work point in the calibrated user coordinate system. Invention Effects

[0014] According to the above method, by using a robot mounted on the trolley for guidance, the trolley can be constructed with minimal equipment and cost while moving along the path with high precision.

[0015] Furthermore, according to the above method, even if the robot deviates slightly from the work position, it can accurately move the tool to the work point on the workpiece to perform the work. Attached Figure Description

[0016] Figure 1 This is a diagram showing the overall structure of the autonomous driving robot system according to the first embodiment. Figure 2 This is a block diagram illustrating the functional structure of the robot control device and the driving control device according to the first embodiment. Figure 3 This is a flowchart illustrating an example of the robot's guidance process for the trolley according to the first embodiment. Figure 4 This is a diagram illustrating an example of the path information involved in the first embodiment. Figure 5 This diagram illustrates the guidance process of the robot according to the first embodiment. Figure 6 This is a flowchart illustrating an example of the trolley's driving control processing according to the first embodiment. Figure 7This is a diagram illustrating the guidance process of the robot involved in a variation of the first embodiment, Example 1. Figure 8 This is a block diagram illustrating the functional structure of the robot control device according to the second embodiment. Figure 9 This is a flowchart illustrating an example of the visual correction processing of the robot according to the second embodiment. Figure 10 This is the first figure used to illustrate the visual correction process of the robot involved in the second embodiment. Figure 11 This is the second figure used to illustrate the visual correction process of the robot involved in the second embodiment. Figure 12 This is the third figure used to illustrate the visual correction processing of the robot involved in the second embodiment. Figure 13 This is the fourth figure used to illustrate the visual correction processing of the robot involved in the second embodiment. Figure 14 This is the fifth figure used to illustrate the visual correction processing of the robot involved in the second embodiment. Figure 15 This is a diagram illustrating an example of the hardware structure of a robot control device and a motion control device. Detailed Implementation

[0017] <First Implementation> The following is for reference Figures 1-6 The implementation method is described in detail.

[0018] (Overall structure) Figure 1 This is a diagram showing the overall structure of the autonomous driving robot system according to the first embodiment. The autonomous driving robot system 100 includes a robot 1, a trolley 2, and an operating PC 3.

[0019] Robot 1 automatically performs tasks instructed by an operator in workspaces such as factories and warehouses. Robot 1 is equipped with a robot control device 10, an arm 11, a vision sensor 12, and a tool 13.

[0020] The robot control device 10 controls the movements of each part of the robot 1. Furthermore, in this embodiment, the robot control device 10 guides the trolley 2 to a designated position. The designated position may be, for example, the work position where the robot 1 performs its tasks, or the storage position for the robot 1 and the trolley 2. The designated position is specified by the operator of the robot 1 via the operation PC3 described later.

[0021] Arm 11 is a multi-joint robotic arm with multiple joints. Arm 11 can arbitrarily change the position and orientation of its front end 11a by rotating each joint.

[0022] A vision sensor 12 is mounted on the front end 11a of the arm 11. The vision sensor 12 is a camera that captures images of the surroundings of the robot 1. Figure 1 An example of a vision sensor 12 equipped with a CCD camera 12A and a 3D camera 12B is shown.

[0023] Tool 13 is detachably mounted on the front end 11a of arm 11. Tool 13 is an instrument used to perform various operations. The operations performed by robot 1 include welding, screw tightening, drilling and other assembly operations, picking operations such as holding and moving parts, finishing operations, ultrasonic inspection operations, and appearance inspection operations. The tool 13 corresponding to the operation is mounted on the front end 11a of arm 11.

[0024] The trolley 2 is, for example, an automated guided vehicle (AGV). The trolley 2 is equipped with a driving control device 20, a drive device 21, driving assistance sensors 22, wheels 23, and stoppers 24.

[0025] The driving control device 20 controls the drive device 21 so that the trolley 2 moves (drives), stops, and changes speed according to the guidance of the robot 1.

[0026] The drive unit 21 is a motor that actuates the wheels 23 of the trolley 2. In this embodiment, the wheels 23 are Mecanum wheels (registered trademark) with multiple rollers, and the drive unit 21 has one motor for each wheel 23. The drive unit 21 changes the combination of rotating wheels 23, the rotational speed of each wheel 23, and the rotational direction according to the control commands of the travel control unit 20. Thus, the trolley 2 can move in any direction. Alternatively, in other embodiments, the wheels 23 may not be Mecanum wheels, but ordinary wheels. In this case, the drive unit 21 has a motor for rotating the wheels and a steering mechanism for changing the orientation of the wheel axles.

[0027] The driving assistance sensor 22 is used to detect the position and direction of travel of the trolley 2, enabling the trolley 2 to move autonomously without guidance from the robot 1. In this embodiment, the driving assistance sensor 22 includes a position detection sensor 22A that detects the amount of movement of the trolley 2 in the horizontal direction (X-axis and Y-axis). The position detection sensor 22A is a sensor that uses the same technology as a so-called optical mouse, detecting the amount of movement of the trolley 2 in the X-axis and Y-axis directions by reading patterns on the ground. In addition, the driving assistance sensor 22 may further include a gyrocompass 22B for detecting the direction of travel of the trolley 2.

[0028] The stop 24 is a device used to fix the trolley 2 in place after it has been moved to the designated position.

[0029] PC3 is the computer operated by the operator of robot 1. PC3 communicates wirelessly with robot 1. PC3 receives commands from the operator and instructs robot 1 on the designated location as the destination, as well as the path (via location, etc.) to the designated location.

[0030] (Functional structure of robot control device) Figure 2 This is a block diagram illustrating the functional structure of the robot control device and driving control device according to the first embodiment. For example... Figure 2 As shown, the robot control device 10 includes a path acquisition unit 101, a sensor information acquisition unit 102, a detection unit 103, a travel instruction unit 104, and an arm control unit 105.

[0031] The path acquisition unit 101 acquires path information D1, which represents the path from the current position of the trolley 2 to the specified position, from the operation PC3.

[0032] The sensor information acquisition unit 102 acquires the image D2 captured by the vision sensor 12. For example, in this embodiment, the sensor information acquisition unit 102 acquires the image D2 captured by the CCD camera 12A of the vision sensor 12 when the trolley 2 is moving.

[0033] The detection unit 103 detects the deviation of the trolley 2 from the designated path in terms of its position and direction (angle) based on the marker L contained in the image captured by the vision sensor 12 (CCD camera 12A). The marker L is, for example, a guide line L1 drawn on the ground, a QR code L2 (registered trademark), or a similar barcode. The marker L is attached to the trolley 2's travel path and the trolley 2's parking position within the work area. Furthermore, in this embodiment, as... Figure 1 The illustration shows an example of the mark L being attached to the upper surface or side of a structure such as the ground or a pillar. In other embodiments, the mark L may also be attached to other locations such as walls, shelves, or ceilings.

[0034] The driving instruction unit 104 outputs driving instructions D4 to the trolley 2 (driving control device 20) to indicate the driving position, stopping, speed, etc. For example, the driving instruction unit 104 instructs the trolley 2 to correct the driving position and direction (angle) of the trolley 2 based on the deviation detected by the detection unit 103.

[0035] The arm control unit 105 controls the arm 11, allowing the vision sensor 12 and tool 13 to be in any position and posture. As described above, in this embodiment, the marker L is attached to the ground. Therefore, during the movement of the trolley 2, the arm control unit 105 controls the arm 11 to be in such a position as... Figure 1 The first pose of the visual sensor 12 is as shown, facing the ground (vertically downwards).

[0036] (Functional structure of the driving control device) like Figure 2 As shown, the driving control device 20 includes an electric motor control unit 201.

[0037] The motor control unit 201 controls the drive unit 21 to move, stop, and change the speed of the trolley 2. When there is no guidance (driving instruction) from the robot 1, the motor control unit 201 controls the drive unit 21 based on the sensor values ​​(movement amount and direction of travel of the trolley 2) of the driving assistance sensor 22 so that the trolley 2 can travel to the designated position.

[0038] Furthermore, when the trolley 2 is moved solely based on the sensor values ​​of the driving assistance sensor 22, the driving position and direction of the trolley 2 may deviate from the designated path due to the condition of the ground in the factory or other environments (such as unevenness or unreadable patterns). Therefore, in this embodiment, the robot control device 10 of the robot 1 detects the deviation of the trolley 2 from the path based on the marker L and issues a driving instruction to the driving control device 20 of the trolley 2 to correct the driving position. The motor control unit 201 of the driving control device 20 controls the drive device 21 according to the driving instruction from the robot control device 10 to adjust the driving position of the trolley 2.

[0039] Furthermore, as described above, the wheels 23 of the trolley 2 in this embodiment are Mecanum wheels. The motor control unit 201 may also include a calculation unit 201A for calculating control command values, which indicate whether the travel motors of each wheel 23 are rotating, their rotation direction, and the number of revolutions. For example, when provided with coordinates as the target position, the calculation unit 201A calculates the control command values ​​of each travel motor required for the trolley 2 to travel to the target position. The motor control unit 201 controls the travel motors based on the control command values ​​calculated by the calculation unit 201A.

[0040] (The robot guides and processes the trolley) Figure 3 This is a flowchart illustrating an example of the robot's guidance process for the trolley according to the first embodiment. Figure 4 This is a diagram illustrating an example of the path information involved in the first embodiment. Figure 5 This diagram illustrates the guidance process of the robot according to the first embodiment. Here, refer to... Figures 3-5 The process of guiding the trolley 2 by the robot control device 10 of robot 1 is described.

[0041] The path acquisition unit 101 acquires path information D1, representing the path from the current position of the trolley 2 to the specified position, from the operation PC3. Figure 4 (Step S101). The specified location is, for example, the working position of robot 1. The specified location and path are input by the operator via operation PC3. Figure 4 As shown, the path information D1 represents the coordinates (X, Y, Z) of the starting position (current position), the passing position and the specified position (destination), as well as the posture (W, P, R) of the trolley 2 at each position.

[0042] If path information D1 is obtained, the robot control device 10 begins guiding the trolley 2 according to path information D1. First, the robot 1 obtains image D2 captured by the vision sensor 12 (step S102). Figure 1 and Figure 5 As shown, in this embodiment, the robot 1 takes an image D2 in a posture (first posture) with the vision sensor 12 facing downward (to the ground) in the vertical direction.

[0043] Next, the detection unit 103 determines whether the marker L is detected in the acquired image D2 (step S103). If the marker L is detected (step S104; yes), the detection unit 103 detects the travel position of the trolley 2 and the amount of deviation relative to the path based on the detection information D3 of the marker L (step S104).

[0044] For example, such as Figure 5 As shown in (a), assume that a guide line L1 as a marker L is detected from image D2 (step S103; yes). Or, as Figure 5 As shown in (c), suppose a QR code L2, which is a marker L, is detected from image D2. In this case, the detection unit 103 detects the deviation amount (step S104) indicating how much the detection position and angle of marker L (L1, L2) deviates from the reference position and reference angle, based on the detection information D3, including the coordinates (X, Y) and angle (angle R around the Z-axis) of the detection position of marker L within image D2. The reference position is the position and angle at which marker L should be detected when the trolley 2 travels correctly along the path. The reference position is provided to the robot 1 in advance.

[0045] Additionally, for example, such as Figure 5As shown in (c), assume that QR code L2, which serves as marker L, is detected from image D2 (step S103; Yes). In this case, the detection unit 103 detects the deviation of the trolley 2 from the QR code L2 in image D2, and detects the driving position based on the reading result of QR code L2 (step S104). For example, QR code L2 records the area names indicating that multiple work areas are configured in the work space. The detection unit 103 is provided with map information that establishes a correspondence between the area names and the positions (XY coordinates) of the work areas, and detects the driving position (X, Y) of the trolley 2 based on the reading result of QR code L2 and the map information. In addition, QR code L2 may also include driving instructions such as "stop", "90° turn", and "driving speed".

[0046] Next, the driving instruction unit 104 outputs driving instruction D4 to the trolley 2 based on the detected deviation and driving position (step S105).

[0047] For example, such as Figure 5 As shown in (a), when a deviation is detected from the guide line L1 in the Y direction, the travel instruction unit 104 outputs a travel instruction D4 to correct the position of the trolley 2 in the Y direction. This travel instruction is represented, for example, by the relative coordinates (X, Y) of the corrected position relative to the current position of the trolley 2. If the trolley 2 can change its direction of travel via the steering mechanism, the relative angle (R) of the corrected direction of travel relative to the current direction of travel can also be included in the travel instruction D4. In addition, the travel instruction unit 104 may also output a travel instruction D4 to correct the travel position of the trolley 2 when the deviation is above a preset allowable value. That is, if the deviation is less than the allowable value, the position correction of the trolley 2 may not be performed.

[0048] In addition, such as Figure 5As shown in (c), when the travel position of the trolley 2 is detected from the QR code L2, the travel instruction unit 104 outputs travel instructions D4 indicating the coordinates of the next target position of the trolley 2, as well as stopping, speed change, etc., based on the travel position and path information D1 of the trolley 2. For example, suppose the path information D1 includes the following path: after moving along the X direction, if a certain work area (area 3) is reached, then move along the Y direction. At this time, if the travel instruction unit 104 detects that the trolley 2 has reached the work area (area 3) based on the reading result of the QR code L2, it outputs travel instructions D4 specifying the coordinates of the next target position to the trolley 2. In addition, if the reading result of the QR code L2 includes information indicating "stop", "90° turn" or "travel speed", the travel instruction unit 104 can include instructions such as stopping, 90° turn, and speed change (acceleration, deceleration) in the travel instructions D4 according to the reading result. Alternatively, when the trolley 2 is stopped, the driving instruction unit 104 detects the deviation of the detection position of the QR code L2, outputs a driving instruction D4 for adjusting the position and posture of the trolley 2, and outputs a driving instruction D4 indicating that the trolley 2 has stopped after adjusting the position and posture of the trolley 2.

[0049] On the other hand, such as Figure 5 As shown in (b), if the detection unit 103 does not detect the mark L from the image D2 (step S103; no), the driving instruction unit 104 outputs a driving instruction D4 to the trolley 2 to indicate autonomous driving (step S106).

[0050] Next, the robot control device 10 determines whether the trolley 2 has stopped at the designated location (step S107). For example, if step S105 indicates that the trolley 2 should stop at the designated location (within the work area as the destination) (step S107; Yes), the robot control device 10 ends the guidance process of the trolley 2. On the other hand, if step S105 does not indicate that the trolley 2 should stop at the designated location (step S107; No), the robot control device 10 returns to step S102 and continues the guidance process of the trolley 2 (steps S102 to S107).

[0051] (Rotary driving control processing) Figure 6 This is a flowchart illustrating an example of the trolley's travel control process according to the first embodiment. Here, refer to... Figure 6 The process of driving control processing of the driving control device 20 of the trolley 2 is explained.

[0052] First, the motor control unit 201 obtains the driving instruction D4 from the robot 1 (step S111).

[0053] Next, the motor control unit 201 determines whether the driving instruction D4 includes an instruction for autonomous driving (step S112).

[0054] When the driving instruction D4 includes an instruction for autonomous driving (step S112; Yes), the motor control unit 201 controls the driving of the vehicle 2 based on the sensor value of the driving assistance sensor 22 (step S113). Autonomous driving control based on sensor values ​​is a known technique, so detailed description is omitted.

[0055] On the other hand, if the driving instruction D4 does not include instructions for autonomous driving (step S112; no), the motor control unit 201 controls the driving of the vehicle 2 according to the content of the driving instruction D4 (step S114). For example, the calculation unit 201A of the motor control unit 201 calculates control command values ​​indicating the rotation direction and number of revolutions of the driving motors (drive devices 21) of each wheel 23 based on the moving target position coordinates, steering, driving speed, and other instructions included in the driving instruction D4. The motor control unit 201 controls the operation of each driving motor based on the calculated control command values. The driving control device 20 executes this repeatedly. Figure 6 The series of processes shown in the diagram cause the trolley 2 to travel to the designated position.

[0056] (Function, effect) As described above, the robot 1 in this embodiment is mounted on a trolley 2 that can move autonomously via a drive device 21. The robot 1 includes: an arm 11 with a vision sensor 12 mounted on its front end 11a, and capable of arbitrarily changing the position and posture of the front end 11a; a detection unit 103 that detects the deviation of the trolley 2's driving position from a specified path based on a marker L contained in an image D2 captured by the vision sensor 12; and a driving instruction unit 104 that instructs the trolley 2 to correct its driving position based on the deviation.

[0057] Thus, by having robot 1 guide trolley 2, it is possible to simultaneously construct trolley 2 with minimal equipment and cost and to move trolley 2 along the path with high precision. Furthermore, since the existing vision sensor 12 of robot 1 can be utilized, the cost of adding additional hardware to robot 1 can be suppressed.

[0058] In addition, the detection unit 103 detects the deviation based on the difference between the position and angle of the marker L contained in the image D2 and the reference position and reference angle.

[0059] With this configuration, robot 1 can accurately detect deviations in the driving position and posture of trolley 2 based on marker L. Therefore, robot 1 can make corrections to ensure that the driving position or posture of trolley 2 follows the path.

[0060] Additionally, L is marked as QR code L2 (two-dimensional code). The detection unit 103 reads information from the QR code L2 contained in the image D2, indicating at least one of the following: driving position, stopping, turning, and driving speed. The driving instruction unit 104 instructs the trolley to indicate at least one of the following: the next moving target position, stopping, turning, and driving speed based on the information read from the QR code L2.

[0061] With this configuration, robot 1 can easily detect the trolley's position based on the information read from QR code L2. Furthermore, if the QR code L2 records information on stopping, turning, and speed, robot 1 can automatically control the movement of trolley 2 based on this information, thus reducing the workload of the operator in pre-entering detailed commands.

[0062] In addition, when the image D2 does not contain the marker L, the driving instruction unit 104 of robot 1 instructs the trolley 2 to drive autonomously to a designated location on the path. The trolley 2 has: a driving assistance sensor 22 capable of detecting the amount of movement and the direction of travel; and a driving control device that enables the trolley 2 to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor 22 when receiving the instruction to drive autonomously.

[0063] With this configuration, the trolley 2 can move autonomously even in areas where the marker L is not attached. Furthermore, upon reaching an area with the marker L, by restarting the guidance of the robot 1, even if the trolley 2 deviates from the path during autonomous driving, it can be corrected to the correct driving position along the path.

[0064] (Variation Example 1) In the first embodiment, an example of the robot 1 taking pictures in a first posture with the vision sensor 12 facing the ground (vertically downward) is described. In this variation, in addition to the first posture, the robot 1 also takes pictures in a second posture with the vision sensor 12 facing forward (horizontally) in the direction of travel.

[0065] Figure 7 This diagram illustrates the guidance process of a robot according to a variation of the first embodiment. For example, as... Figure 7 As shown in (a), when the detection unit 103 cannot detect the mark L in the first posture, the arm control unit 105 changes the position and posture of the arm 11 to a second posture in which the vision sensor 12 is oriented towards the front of the travel direction. Figure 1As shown in the example, the marker L has a QR code L2a attached to the upper surface of the support and a QR code L2b attached to the side. When a support with such a QR code L2b is provided in front of the travel direction, the detection unit 103 can detect the QR code L2b from the image D2 in a second posture. The support is located on or near a guide line L1 attached to the ground.

[0066] When the detection unit 103 detects the QR code L2b in the second posture ( Figure 3 Step S103; Yes), based on the detection information D3 of QR code L2b, the driving position of trolley 2 and the deviation relative to the path are detected. Figure 3 Step S104). Additionally, detection information D3 is as follows: Figure 7 As shown in the example, the detection information includes the coordinates (X, Y, Z), angle (W, P, R), and detection dimensions of the QR code L2b's detection position. Furthermore, the detection information D3 may also include the reading result of the QR code L2b. With this configuration, the robot control device 10 switches to a second posture in areas where the ground is not marked L, enabling it to detect the QR code L2b ahead in the direction of travel and continue guiding the trolley 2.

[0067] In addition, such as Figure 7 As shown in (b), when the detection size of the QR code L2b in the second posture is greater than or equal to a predetermined size, that is, when the vision sensor 12 is sufficiently close to the support, the arm control unit 105... Figure 7 (c) shows the switch to the first posture. As described above, the support is positioned on or near the mark L attached to the ground, so returning to the first posture after sufficiently approaching the support allows for immediate detection of the mark L attached to the ground. In the first posture, the distance between the vision sensor 12 and the mark L is fixed, thus improving the detection accuracy of the deviation in the detection unit 103 compared to the second posture. In this way, the robot 1 switches between the first posture and the second posture based on the detection state of the mark L, thereby maintaining a high detection accuracy of the deviation in the first posture in areas where the first mark L2a is attached to the ground, and continuously guiding the trolley 2 in the second posture in areas where the first mark L2a is not attached to the ground. That is, the period during which the trolley 2 performs autonomous driving with lower accuracy can be reduced, thus enabling the trolley 2 to travel along the path more reliably.

[0068] (Variation Example 2) In the first embodiment, an example was described where the detection unit 103 uses guide lines L1 and QR codes L2 as markers L to detect the travel position of the trolley 2. In this modified example, the detection unit 103 uses structures within the work area (including the area where the trolley 2 travels, stops, etc.) as markers L to detect the travel position of the trolley 2. These structures include, for example, shelves, signs, products, fixtures, walls, pillars, and other parts of a building. Furthermore, in this modified example, the robot 1 always takes pictures in a second posture with the vision sensor 12 facing forward (horizontally) in the direction of travel.

[0069] Inspection Department 103 Figure 3 In step S103, the shape (three-dimensional shape or two-dimensional shape) of the structure is detected from the image D2 captured by the vision sensor 12, and compared with reference data that has been pre-recorded as the shape and position of each structure as a marker L, thereby determining whether the marker L has been detected from the image D2.

[0070] Next, the testing department 103... Figure 3 In step S104, the traveling position of the trolley 2 is detected based on the marker L (structure) detected from image D2. Furthermore, the detection unit 103 detects the deviation of the trolley 2 relative to a reference position based on the detected position of the marker L (structure) in image D2. The detection unit 103 compares the shape of the marker L detected from image D2 with reference data (the three-dimensional shape of the marker L, or the two-dimensional shape of the marker L observed from various directions), detects the relative angle between the traveling direction of the trolley 2 and the marker L, and detects the deviation of this relative angle relative to a reference angle. The reference position and reference angle are preset to indicate the position and relative angle at which the marker L should be detected when the trolley 2 travels correctly along the path.

[0071] Driving indicator 104 Figure 3 In step S105, based on the driving position and deviation detected by the detection unit 103, a driving instruction D4 is output to the trolley 2. In addition, if the reference data records driving instructions (such as "stop", "90° turn", "driving speed", etc.) to be implemented when the marker L is detected, the driving instruction unit 104 can include the driving instructions read from the reference data in the driving instruction D4 for the trolley 2.

[0072] With this configuration, even in environments where the white lines L1 and QR codes L2 attached to the ground are obscured due to products, fixtures, etc., robot 1 can detect the traveling position of trolley 2 and its deviation from the path based on the shape of the structures in the work area, and guide trolley 2 to travel along the path. Robot 1 can be configured to switch between the functions of the first embodiment or modification 1 and modification 2 in a way that matches the environment of the work area.

[0073] <Second Implementation> The following is for reference Figure 8 The second embodiment will be described. The same symbols used in the structure of the second embodiment as in the first embodiment are omitted from the description. After moving to the work position (designated position), robot 1 performs operations on the workpiece.

[0074] (Functional structure of robot control device) Figure 8 This is a block diagram illustrating the functional structure of the robot control device according to the second embodiment. For example... Figure 8 As shown, the robot control device 10 according to this embodiment also includes a vision correction unit 106 and a tool control unit 107.

[0075] The vision correction unit 106 corrects the user coordinate system of the workpiece based on the image captured by the vision sensor 12, matching the actual position and posture of the workpiece.

[0076] After the arm control unit 105 moves the tip 11a (tool 13) of the arm 11 to the workpiece's working position, the tool control unit 107 controls the tool 13 to perform a predetermined operation corresponding to the tool 13. Predetermined operations include, for example, welding, screw tightening, drilling, and other assembly operations; part handling and movement, picking operations; finishing operations; ultrasonic inspection; and visual inspection. The tool control unit 107 has an application program for controlling various tools 13. Furthermore, this application program can be added or replaced arbitrarily when changing tools 13.

[0077] The arm control unit 105 moves the tool 13 to the working position on the workpiece based on the user coordinate system corrected by the vision correction unit 106.

[0078] (Visual correction processing) Figure 9 This is a flowchart illustrating an example of the visual correction processing of the robot according to the second embodiment. Figures 10-14 These are the first to fourth figures used to illustrate the visual correction processing of the robot according to the second embodiment. Hereinafter, refer to... Figures 9-14 The process of visual correction processing for robot control device 10 is explained.

[0079] First, the operator arbitrarily determines the reference position UF (origin) of the user coordinate system representing the coordinates of the work object of robot 1 (step S201). The vision correction unit 106 sets the position specified by the operator through operation PC3 as the reference position UF. The reference position UF is represented by a vector (X, Y, Z, W, P, R) representing the position and direction from the origin P0 of robot 1.

[0080] Next, the operator performs a teaching operation on the UF (step S202). For example, such as... Figure 10 As shown, the robot 1 uses its vision sensor 12 to capture an image of the first workpiece W1. While viewing the image, the operator specifies the position for the tool 13 to perform the operation. The vision correction unit 106 stores the specified position as a teaching point TP. Figure 10 In the example, five locations are set as teaching points TP1 to TP5.

[0081] Additionally, the operator sets a representative position for the workpiece as observed from the robot 1 (step S203). For example, the operator designates any position P1 of the first workpiece W1 as the representative position. The representative position is selected from characteristic locations of the first workpiece W1, such as locations with screw holes, protrusions, etc. The vision correction unit 106 sets the position P1 designated by the operator as the representative position. Furthermore, the operator... Figure 11 As shown in the example, multiple positions P1, P2, and P3 can also be set. In this case, the visual correction unit 106 sets the average position of these positions P1, P2, and P3 as the representative position. Here, for the sake of simplicity, an example with P1 as the representative position will be described.

[0082] Next, the vision correction unit 106 sets the reference position UF of the user coordinate system and the representative point of the workpiece observed from the robot to a known state. Specifically, the vision correction unit 106 as follows: Figure 11 As shown, calculate the vector Vn from the representative position (e.g., P1) to the reference position UF (step S204).

[0083] After setting the vector Vn, the second category and subsequent workpieces (second workpiece W2) are set, and images are taken using the vision sensor 12 of robot 1. For example, as... Figure 12 As shown, the position and orientation of the second workpiece W2 are sometimes different from those of the first workpiece W1. Therefore, as... Figure 12 As shown in the example, the teaching points TP1 to TP5 set for the first workpiece W1 will have a large deviation on the second workpiece W2. Therefore, the vision correction unit 106 of this embodiment corrects the user coordinate system to match the position and posture of the second workpiece W2 for subsequent operations. Specifically, firstly, the operator, such as Figure 13As shown, while viewing the captured image of the second workpiece W2, a representative position P1' on the second workpiece W2 corresponding to the representative position P1 of the first workpiece W1 is designated (step S205). The vision correction unit 106 sets the position designated by the operator as the representative position P1' of the second workpiece. Furthermore, if the operator designated multiple positions P1 to P3 of the first workpiece W1 in step S203, multiple positions P1' to P3' are also designated for the second workpiece W2. In this case, the vision correction unit 106 sets the average position of these positions P1' to P3' as the representative position of the second workpiece W2.

[0084] Next, the vision correction unit 106 uses the inverse matrix of the representative position P1' of the second workpiece W2 and the vector Vn to calculate the corrected reference position UF', which has shifted the reference position UF of the user coordinate system (step S206). To calculate the corrected reference position UF', it is only necessary to calculate the vector Vb from the origin P0 to the corrected reference position UF' based on the vector Va from the origin P0 of the robot 1 to the representative position P1' and the inverse matrix of the vector Vn. Vectors Va and Vb are vectors (X, Y, Z, W, P, R) representing the position and direction from the origin P0.

[0085] Additionally, the visual correction unit 106, as Figure 14 As shown, using the reference position UF', all teaching points TP1 to TP5 specified in step S202 are simultaneously shifted to working points TP1' to TP5' on the second workpiece W2 (step S207). After obtaining the working points TP1' to TP5' for the second workpiece W2, the arm control unit 105 moves the tool tip point TCP sequentially to each working point. In this way, whenever the tool tip point TCP reaches each working point, the tool control unit 107 controls the movement of the tool 13 to perform the predetermined operation on the second workpiece W2. Furthermore, this is also performed on subsequent second workpieces W2. Figure 9 Steps S205 to S207 are used to determine the working points TP1' to TP5' for each second workpiece W2 before proceeding with the operation.

[0086] (Function, effect) As described above, the robot 1 according to this embodiment also includes a vision correction unit 106, which corrects the user coordinate system of the workpiece based on the image captured by the vision sensor 12 and matches the actual position and posture of the workpiece. Furthermore, the arm control unit 105 controls the position and posture of the arm 11, so that the tool tip point TCP mounted on the front end 11a of the arm 11 moves to the work point in the corrected user coordinate system.

[0087] With this configuration, even if the position and orientation of the second workpiece W2 are different each time a new workpiece (second workpiece W2) is set, robot 1 can still move the tool tip point TCP to the correct work point. This improves the accuracy of robot 1's work. Furthermore, even if the position and orientation of the carriage 2 deviate from the designated work position, robot 1 can still accurately move the tool tip point TCP to the work point on the second workpiece W2 to perform the work. Therefore, the guidance process for carriage 2 in the first embodiment can tolerate errors in the position and orientation of carriage 2 relative to the work position. Corresponding to this tolerance, the process of fine-tuning the position and orientation of carriage 2 at the work position can be omitted, thus shortening the time required for the movement and position adjustment of robot 1 and carriage 2.

[0088] <Hardware Structure> Figure 15 This diagram illustrates an example of the hardware structure of a robot control device and a motion control device. The computer 900 includes a CPU 901, a main storage device 902, an auxiliary storage device 903, an input / output interface 904, and a communication interface 905.

[0089] The robot control device 10 and the driving control device 20 are respectively installed in the computer 900. Furthermore, the aforementioned functions are stored as programs in the auxiliary storage device 903. The CPU 901 reads the program from the auxiliary storage device 903, expands it in the main storage device 902, and executes the aforementioned processing according to the program. Additionally, the CPU 901 secures a storage area in the main storage device 902 according to the program. Furthermore, the CPU 901 secures a storage area in the auxiliary storage device 903 according to the program to store the data being processed.

[0090] Alternatively, the program used to implement all or part of the functions of the robot control device 10 and the driving control device 20 can be recorded in a computer-readable recording medium, allowing the computer system to read and execute the program recorded in the recording medium, thereby performing processing of each functional unit. The term "computer system" here includes hardware such as the operating system and peripheral devices. Furthermore, if the "computer system" uses a WWW system, it also includes a homepage providing environment (or display environment). Additionally, "computer-readable recording medium" refers to removable media such as CDs, DVDs, and USB drives, and storage devices such as hard drives built into the computer system. Furthermore, when the program is delivered to the computer 900 via a communication line, the receiving computer 900 can expand the program in the main storage device 902 and execute the aforementioned processing. Moreover, the program can be used to implement a portion of the aforementioned functions, and it can also be used to implement the aforementioned functions through combination with programs already recorded in the computer system.

[0091] <Other Implementation Methods> The embodiments have been described in detail above with reference to the accompanying drawings, but the specific structure is not limited to the above description and various design changes are possible. That is, in other embodiments, the order of the above processes can also be appropriately changed. In addition, some processes can be executed in parallel.

[0092] <Postscript> The robot, autonomous driving robot system, guidance method, control method, and program described in the above embodiments can be understood as follows, for example.

[0093] (1) According to the first method, a robot 1 is mounted on a trolley 2 capable of autonomous driving. The robot 1 includes: an arm 11 capable of arbitrarily changing the position and posture of its front end; a vision sensor 12 mounted on the front end of the arm 11; a detection unit 103 that detects the deviation of the trolley 2 relative to a specified path based on a marker L contained in an image captured by the vision sensor 12; and a driving instruction unit 104 that instructs the trolley 2 to correct its driving position or driving posture based on the deviation.

[0094] Thus, by having robot 1 guide trolley 2, it is possible to simultaneously construct trolley 2 with minimal equipment and cost and to move trolley 2 along the path with high precision. Furthermore, since the existing vision sensor 12 of robot 1 can be utilized, the cost of adding additional hardware to robot 1 can be suppressed.

[0095] (2) According to the second method, in the robot 1 involved in the first method, the detection unit 103 detects the deviation based on the difference between the position and angle of the marker L contained in the image and the reference position and reference angle.

[0096] With this configuration, robot 1 can accurately detect deviations in the driving position and posture of trolley 2 based on marker L. Therefore, robot 1 can make corrections to ensure that the driving position or posture of trolley 2 follows the path.

[0097] (3) According to the third method, in the robot 1 involved in the first or second method, the marker L is a QR code L2, the detection unit 103 reads information from the QR code L2 contained in the image indicating at least one of the driving position, parking, turning and driving speed, and the driving instruction unit 104 instructs the trolley 2 on the next moving target position, parking, turning and driving speed based on the information read from the QR code L2.

[0098] With this configuration, robot 1 can easily detect the trolley's position based on the information read from QR code L2. Furthermore, if the QR code L2 records information about stopping, turning, and speed, robot 1 can automatically control the movement of trolley 2 based on this information, thus reducing the workload of the operator in pre-entering detailed commands.

[0099] (4) According to the fourth method, in the robot 1 involved in the first or second method, L is marked as a structure in the working area of ​​the trolley 2. The detection unit 103 compares the reference data that records the shape of the structure and information indicating at least one of the driving position, stopping, turning and driving speed with the shape detected from the image D2 to detect at least one of the information. Based on the detected information, the driving instruction unit 104 instructs the trolley 2 to indicate at least one of the next moving target position, stopping, turning and driving speed.

[0100] With this configuration, even in environments where the white line L1 and QR code L2 attached to the ground are obscured due to products, fixtures, etc., robot 1 can detect the driving position of trolley 2 and whether it needs to stop based on the shape of the structures in the work area, and guide trolley 2 appropriately.

[0101] (5) According to the fifth method, the robot 1 involved in any of the first to third methods further includes an arm control unit 105 that controls the position and posture of the front end of the arm 11. The marker L has a first marker L2a attached in a vertical direction and a second marker L2b attached in a horizontal direction. When the trolley 2 is moving, the arm control unit 105 controls the arm 11 to make it a first posture in which the vision sensor 12 is oriented vertically and can capture the first marker L2a. When the trolley 2 is moving and the image captured in the first posture does not contain the first marker L2a, the arm control unit 105 controls the arm 11 to make it a second posture in which the vision sensor 12 is oriented horizontally and can capture the second marker L2b.

[0102] With this configuration, even in areas where the first marker L2a is not present on the ground, robot 1 can switch to a second posture and detect the deviation based on the second marker L2b, thus enabling continuous guidance of the trolley 2. In other words, it reduces the period during which the trolley 2 performs autonomous driving with lower precision.

[0103] (6) According to the sixth method, in the robot 1 involved in the fifth method, the detection unit 103 also detects the size of the second mark L2b when the arm 11 is in the second posture, and the arm control unit 105 switches the arm 11 from the second posture to the first posture when the detected size of the second mark L2b is above a predetermined size.

[0104] In this way, robot 1 can maintain a high detection accuracy of the first mark L2a in the area on the ground where the first mark L2a is attached, while maintaining a first posture. As a result, trolley 2 can travel along the path more reliably.

[0105] (7) According to the seventh method, the robot 1 involved in any of the first to sixth methods further includes: a vision correction unit 106, which corrects the user coordinate system of the workpiece based on the image captured by the vision sensor 12 and matches the actual position and posture of the workpiece; and an arm control unit 105, which controls the position and posture of the arm 11 so that the tip of the tool mounted on the front end of the arm 11 moves to the work point in the corrected user coordinate system.

[0106] With this configuration, even if the position and orientation of the second workpiece W2 are different each time a new workpiece (second workpiece W2) is set, robot 1 can still move the tool tip point TCP to the correct work point. This improves the accuracy of robot 1's work. Furthermore, even if the position and orientation of the carriage 2 deviate from the designated work position, robot 1 can still accurately move the tool tip point TCP to the work point on the second workpiece W2 to perform the work. Therefore, the guidance process for carriage 2 in the first embodiment can tolerate errors in the position and orientation of carriage 2 relative to the work position. Corresponding to this tolerance, the process of fine-tuning the position and orientation of carriage 2 at the work position can be omitted, thus shortening the time required for the movement and position adjustment of robot 1 and carriage 2.

[0107] (8) According to the eighth method, a robot 1 automatically performs a predetermined operation on a workpiece. The robot 1 includes: an arm 11 capable of arbitrarily changing the position and posture of its front end; a vision sensor 12 mounted on the front end of the arm 11; a tool mounted on the front end of the arm 11; a vision correction unit that corrects the user coordinate system of the workpiece based on the image captured by the vision sensor 12 and matching it with the actual position and posture of the workpiece; and an arm control unit 105 that controls the position and posture of the arm 11 so that the tip of the tool mounted on the front end of the arm 11 moves to the work point in the corrected user coordinate system.

[0108] With this configuration, even if the position and orientation of the second workpiece W2 are different each time a new workpiece (second workpiece W2) is set, robot 1 can still move the tool tip point TCP to the correct work point. This improves the accuracy of robot 1's work. Furthermore, even if the position and orientation of the carriage 2 deviate from the designated work position, robot 1 can still accurately move the tool tip point TCP to the work point on the second workpiece W2 to perform the work. Therefore, the guidance process for carriage 2 in the first embodiment can tolerate errors in the position and orientation of carriage 2 relative to the work position. Corresponding to this tolerance, the process of fine-tuning the position and orientation of carriage 2 at the work position can be omitted, thus shortening the time required for the movement and position adjustment of robot 1 and carriage 2.

[0109] (9) According to the ninth method, an autonomous driving robot system 100 includes a trolley 2 and a robot 1 involved in any of the first to eighth methods. In the autonomous driving robot system 100, the driving instruction unit 104 of the robot 1 instructs the trolley 2 to drive autonomously when the image does not contain the marker L. The trolley 2 has: a driving assistance sensor 22 that can detect the amount of movement and the direction of travel; and a driving control device 20 that, upon receiving the instruction to drive autonomously, enables the trolley 2 to drive autonomously based on the amount of movement and the direction of travel detected by the driving assistance sensor 22.

[0110] With this configuration, the trolley 2 can move autonomously even in areas where the marker L is not attached. Furthermore, upon reaching an area with the marker L, by restarting the guidance of the robot 1, even if the trolley 2 deviates from the path during autonomous driving, it can be corrected to the correct driving position along the path.

[0111] (10) According to the tenth method, a guidance method is a guidance method for a trolley 2, the trolley 2 uses a robot 1, the robot 1 is mounted on the trolley 2 which is capable of autonomous driving, and has an arm 11 which can arbitrarily change the position and posture of the front end and a vision sensor 12 installed at the front end of the arm 11. The guidance method has the following steps: based on the marker L contained in the image captured by the vision sensor 12, to detect the amount of deviation of the trolley 2 relative to the specified path; and based on the amount of deviation, to instruct the trolley 2 to correct the driving position or driving posture of the trolley 2.

[0112] (11) According to the eleventh method, a procedure is used to cause a robot 1 mounted on a trolley 2 capable of autonomous driving and having an arm 11 capable of arbitrarily changing the position and posture of its front end and a vision sensor 12 mounted on the front end of the arm 11 to perform the following steps: detecting the amount of deviation of the trolley 2 relative to a specified path based on a marker L contained in an image captured by the vision sensor 12 mounted on the front end of the arm 11 of the robot 1; and instructing the trolley 2 to correct the driving position or driving posture of the trolley 2 based on the amount of deviation.

[0113] (12) According to the twelfth method, a control method is a control method for a robot 1, wherein the robot 1 has an arm 11 capable of arbitrarily changing the position and posture of its front end, a vision sensor 12 and a tool installed at the front end of the arm 11, and performs a predetermined operation on the workpiece. The control method has the following steps: based on the image captured by the vision sensor 12, the user coordinate system of the workpiece is corrected to match the actual position and posture of the workpiece; and the position and posture of the arm 11 are controlled so that the tip of the tool moves to the work point in the corrected user coordinate system.

[0114] (13) According to the thirteenth method, a program enables a robot 1 having an arm 11 capable of arbitrarily changing the position and posture of its front end, a vision sensor 12 and a tool mounted on the front end of the arm 11, and performing a predetermined operation on a workpiece to perform the following steps: calibrating the user coordinate system of the workpiece based on an image captured by the vision sensor 12 mounted on the front end of the arm 11 of the robot 1, matching the actual position and posture of the workpiece; and controlling the position and posture of the arm 11 so that the tip of the tool mounted on the front end of the arm 11 moves to the work point in the calibrated user coordinate system. Industrial applicability

[0115] According to the above method, by using a robot mounted on the trolley for guidance, the trolley can be constructed with minimal equipment and cost while moving along the path with high precision.

[0116] Furthermore, according to the above method, even if the robot deviates slightly from the work position, it can accurately move the tool to the work point on the workpiece to perform the work. Symbol Explanation

[0117] 100 Autonomous Driving Robot System 1. Robot 10. Robot control device 101 Path Acquisition Department 102 Sensor Information Acquisition Unit 103 Testing Department 104 Driving Instruction Section 105 Arm Control Unit 106 Vision Correction Department 107 Tool Control Department 11 arms 12 visual sensors 12A CCD camera 12B 3D camera 13 Tools 2 cars 20. Driving control device 201 Electric Motor Control Section 201A Computing Department 21. Drive unit 22 Driving assistance sensors 22A Position Detection Sensor 22B Gyroscope Compass 23 wheels 24 Stops 3. Operating the PC L mark L1 guide line (marker) L2 QR code (QR code, marker) L2a QR code (first marker) L2b QR code (second marker).

Claims

1. A robot mounted on a trolley capable of autonomous movement, said robot comprising: The arm can freely change the position and posture of its front end; A vision sensor is mounted on the front end of the arm; The detection unit detects the amount of deviation of the trolley from the specified path based on the markers contained in the image captured by the vision sensor. as well as The driving instruction unit instructs the trolley to correct its driving position or driving posture based on the deviation amount.

2. The robot according to claim 1, wherein, The detection unit detects the deviation based on the difference between the position and angle of the marker contained in the image and the reference position and reference angle.

3. The robot according to claim 1 or 2, wherein, The mark is a QR code. The detection unit reads information from the QR code contained in the image, indicating at least one of the following: driving position, parking, turning, and driving speed. The driving instruction unit, based on the information read from the QR code, instructs the trolley to at least one of the following: the next moving target position, stopping, turning, and driving speed.

4. The robot according to claim 1 or 2, wherein, The markings refer to the structures within the operating area of ​​the trolley. The detection unit compares reference data, which records the shape of the structure and information indicating at least one of driving position, parking, turning, and driving speed, with the shape detected from the image to detect at least one of the information. Based on the detected information, the driving instruction unit instructs the trolley to at least one of the following: the next moving target position, stopping, turning, and driving speed.

5. The robot according to claim 1 or 2, wherein, The robot also has an arm control unit that controls the position and posture of the front end of the arm. The mark has a first mark attached in a vertical orientation and a second mark attached in a horizontal orientation. The arm control unit controls the arm during the movement of the trolley to achieve a first posture in which the vision sensor is oriented vertically to capture the first mark. The arm control unit controls the arm while the trolley is moving and the image captured in the first posture does not contain the first mark, so that it is in a second posture in which the vision sensor is oriented horizontally and can capture the second mark.

6. The robot according to claim 5, wherein, The detection unit also detects the size of the second mark when the arm is in the second posture. When the detected size of the second mark is greater than a predetermined size, the arm control unit switches the arm from the second posture to the first posture.

7. The robot according to claim 1 or 2, wherein, The robot also has the following features: The vision correction unit, based on the images captured by the vision sensor, corrects the user coordinate system of the workpiece to match its actual position and posture; and The arm control unit controls the position and posture of the arm, so that the tip of the tool mounted at the front end of the arm moves to the work point in the calibrated user coordinate system.

8. A robot that automatically performs predetermined operations on a workpiece. The robot has the following features: The arm can freely change the position and posture of its front end; A vision sensor is mounted on the front end of the arm; A tool is mounted on the front end of the arm; The vision correction unit corrects the user coordinate system of the workpiece by matching the actual position and posture of the workpiece with the image captured by the vision sensor. as well as The arm control unit controls the position and posture of the arm, so that the tip of the tool mounted at the front end of the arm moves to the work point in the calibrated user coordinate system.

9. An autonomous driving robot system, comprising a trolley and the robot as described in claim 1 or 2. The robot's driving indicator unit instructs the trolley to drive autonomously when the image does not contain the marker. The trolley has the following features: Driving assistance sensors can detect the amount of movement and direction of travel; and Upon receiving the instruction for autonomous driving, the driving control device enables the trolley to drive autonomously based on the amount of movement and direction of travel detected by the driving assistance sensor.

10. A guidance method for a trolley, the trolley using a robot, the robot being mounted on the autonomously moving trolley and having an arm capable of arbitrarily changing the position and posture of its front end, and a vision sensor mounted on the front end of the arm. The guiding method includes the following steps: The deviation of the trolley from a specified path is detected based on markers contained in images captured by the vision sensor; and Based on the deviation, the trolley is instructed to correct its driving position or driving posture.

11. A program that causes a robot mounted on an autonomously moving trolley, having an arm capable of arbitrarily changing the position and orientation of its front end, and a vision sensor mounted on the front end of said arm, to perform the following steps: The deviation of the trolley from a designated path is detected based on markers contained in images captured by a vision sensor mounted at the front end of the robot's arm; and Based on the deviation, the trolley is instructed to correct its driving position or driving posture.

12. A control method for a robot, wherein the robot has an arm capable of arbitrarily changing the position and posture of its front end, and a vision sensor and tools mounted on the front end of the arm, and performs predetermined operations on a workpiece. The control method comprises the following steps: Based on the images captured by the vision sensor, the user coordinate system of the workpiece is corrected to match the actual position and posture of the workpiece; and Control the position and posture of the arm so that the tip of the tool moves to the work point in the calibrated user coordinate system.

13. A program that enables a robot having an arm capable of arbitrarily changing the position and orientation of its front end, and a vision sensor and tool mounted on the front end of said arm, and performing a predetermined operation on a workpiece, to perform the following steps: The user coordinate system of the workpiece is corrected based on images captured by a vision sensor mounted at the front end of the robot's arm to match the actual position and posture of the workpiece; and Control the position and posture of the arm so that the tip of the tool mounted at the front end of the arm moves to the work point in the calibrated user coordinate system.

Images